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EXECUTIVE  SUMMARY 


The  VSSC  (Weapon  System  Support  Cost)  subsystem  of  the  Air  Force 
VAMOSC  system  collects  operating  and  support  costs  for  Air  Force  weapon 
systems.  This  report  presents  an  analysis  and  discussion  of  a  user  sur¬ 
vey,  conducted  by  Desmatics,  Inc.  in  early  1984,  designed  to  obtain  an 
evaluation  of  WSSC  by  current  users  and  to  gather  information  about 
potential  users. 

The  survey  consisted  of  a  total  of  26  questions  of  three  types: 
multiple  choice,  rating  scale,  and  open  ended.  In  all,  293  usable 
surveys  were  returned,  of  which  55  were  from  current  WSSC  users.  A 
discussion  of  the  responses  to  each  survey  question  is  given  in  the 
report.  Based  on  the  analysis  of  the  survey  results,  a  number  of  con¬ 
clusions  were  reached.  In  general,  respondents  gave  a  positive  evalu¬ 
ation  of  the  WSSC  system,  but  at  the  same  time  indicated  a  relatively 
low  level  of  familiarity  with  WSSC. 

In  addition  to  discussing  the  conclusions  resulting  from  the  analysis 
of  the  survey  results,  Desmatics  sets  forth  a  number  of  reconmendations 
for  consideration  by  the  Office  of  VAMOSC.  These  deal  with  publicizing 
the  WSSC  system,  implementing  changes  or  enhancements,  revising  the  users 
manual,  and  conducting  a  follow-up  survey  in  approximately  three  years. 
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I.  INTRODUCTION 


Desmatics,  Inc.,  under  Contract  No.  F33600-80-C-0554,  is  con¬ 
ducting  an  evaluation  of  the  cost  allocation  algorithms  employed  in 
the  Weapon  System  Support  Cost  (WSSC)  subsystem  of  VAMOSC,  the  Air 
Force  Visibility  and  Management  of  Operating  and  Support  Costs  system. 
As  part  of  this  contract  Desmatics  was  tasked  with  conducting  a  WSSC 
system  user  acceptability  study. 

The  Statement  of  Work  for  this  task  sets  forth  the  following 
objectives: 

1.  Determine  the  nature  and  extent  of  the  current  and 
potential  WSSC  system  user  community. 

2.  Conduct  research  with  members  of  the  defined  user 
community  to  measure  the  degree  to  which  the  system 
products  are  utilized  and  meet  the  requirements  of 
particular  individual  users  or  types  of  users. 

3.  Identify  changes  which  could  enhance  the  utility  of 
the  system  to  the  user  community  and  propose  methods 
for  achieving  these  enhancements. 

The  purpose  of  this  report  is  to  describe  the  methodology  used  to  con¬ 
duct  the  WSSC  User  Acceptability  Study,  to  evaluate  the  study  results, 
and  to  provide  recommendations  for  enhancing  the  utility  of  the  WSSC 
system  to  the  user  community. 

VAMOSC  is  an  AF  management  information  system  composed  of  three 
major  modules: 

(1)  C-E  (Ground  Communications-Electronics)  [3]  which  pro¬ 
vides  costs  for  ground  C-E  equipment  at  the  Type  Model 
Series  (TMS)  level. 
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(2)  CSCS  (Component  Support  Cost  System)  [A]  which  provides 
visibility  of  maintenance  and  support  costs  for  aircraft 
(and  associated  engines)  subsystems  and  components, 

and  (3)  WSSC  (Weapon  System  Support  Cost)  [2]  which  collects, 

computes,  and  displays  costs  for  aircraft  at  the  Mission 
Design  Series  (MDS)  level. 

The  data  system  designators  (DSDs)  for  the  three  systems  are 
D160A,  D160B  and  D160C,  respectively.  Another  module  called  VAMOH 
(VAMOSC  Overhead),  DSD  D160.,  is  a  subsystem  which  preprocesses  selected 
data  [ 1] .  Operating  and  support  cost  data  reported  by  the  WSSC  system, 
the  subject  of  this  user  acceptability  study,  is  intended  for  use  in 
life  cycle  costing,  design  studies,  design  trade-off  studies,  and 
logistics  policy  guidance  for  AF  weapon  systems  [2]. 


II.  METHODOLOGY 


This  section  describes  the  methodology  used  for  the  Desmatics 
WSSC  User  Acceptability  Study.  The  study  involved  definition  of  the 
user  community,  background  research,  survey  design,  administration  of 
the  survey  to  members  of  the  defined  user  community,  and  analysis  of 
results. 

A.  WSSC  USER  COMMUNITY 


The  WSSC  system  user  community  is  defined  as  including  both  current 
and  potential  users  of  the  system.  Determining  the  nature  and  extent 
of  this  user  community  was  an  ongoing  process  during  the  acceptability 
study.  It  consisted  of  the  compilation  of  a  list  of  specific  individ¬ 
uals/offices  who  are  current  or  potential  users.  Considerable  effort 
was  expended  in  making  this  list  as  complete  as  possible.  The  two 
major  objectives  for  building  this  list  of  users/potential  users  were: 

(1)  to  locate  these  individuals  in  order  to  later  obtain  their  opinions 
of  the  WSSC  system  (and  suggestions  for  improvements)  through  a  written 
survey,  and  (2)  to  create  further  awareness  of  the  system  and  its  capa¬ 
bilities  in  the  community  at  large. 

Desmatics'  initial  list  of  users/potential  users  consisted  of  the 
names  of  invitees  to,  and  attendees  of,  the  VAMOSC  User/FOE  (Final  Opera¬ 
tional  Evaluation)  Conference  held  on  12-13  April  1983  in  Dayton,  Ohio. 

To  this  was  added  the  list  of  all  individuals  who  had  requested  WSSC 
data.  Throughout  the  study  the  Office  of  VAMOSC  continued  to  supply 


Desmatics  with  the  names  and  addresses  of  all  individuals  requesting 
data. 

As  part  of  the  background  research  in  developing  the  written  sur¬ 
vey,  Desmatics  conducted  telephone  interviews  with  a  number  of  users. 
These  interviews  were  also  used  to  obtain  names  of  users/potential 
users.  Finally,  in  order  to  locate  additional  users /potential  users 
within  the  AF  community,  a  letter  prepared  with  the  assistance  of  the 
Office  of  VAMOSC  and  HQ  USAF/LEYM  was  mailed  to  approximately  300  Air 
Force  office  addresses  accumulated  from  previous  Office  of  VAMOSC  dis-  . 
tribution  lists.  This  letter  requested  the  names  and  addresses  of  all 
individuals  within  the  office,  or  other  individuals  (including  contrac¬ 
tors),  known  to  be  current  or  potential  WSSC  users.  Of  the  300  letters 
mailed,  147  were  returned.  A  copy  of  the  letter  is  in  Appendix  A. 

B.  SURVEY  DEVELOPMENT 

Background  information  necessary  to  design  the  written  survey  was 
obtained  in  two  ways.  First,  Desmatics  assisted  the  Office  of  VAMOSC 
with  the  design  and  analysis  of  the  WSSC  and  C-E  user  surveys  adminis¬ 
tered  at  the  1983  VAMOSC  User/FOE  Conference.  Second,  Desmatics  con¬ 
ducted  telephone  interviews  with  twelve  WSSC  users.  Their  names  were 
selected  from  the  1983  VAMOSC  User/FOE  Conference  attendee  list  and 
from  the  list  of  users  who  had  requested  data  from  the  Office  of 
VAMOSC.  Six  of  these  individuals  were  from  the  Air  Force  community  and 
six  were  from  the  contractor  community.  The  interviews  were  structured 
informally  around  a  list  of  questions  designed  to  determine  such  things 


as  how  WSSC  data  was  being  used  and  the  perceived  quality  of  WSSC  data, 
the  user's  manual,  and  the  training  provided.  Suggestions  for  additions 
or  improvements  were  also  solicited.  All  interviewees  were  cooperative 
and  helpful. 

Questions  for  the  written  survey  were  developed  from  the  infor¬ 
mation  acquired  from  the  activities  described  above.  The  survey  was 
constructed  through  a  series  of  drafts  which  were  reviewed  internally 
and  revised.  A  draft  was  then  submitted  to  the  Office  of  VAMOSC  for 
approval,  and  after  minor  modifications  the  final  version  was  prepared. 


III.  WSSC  USER  SURVEY 


The  final  phase  of  the  WSSC  user  acceptability  study  consisted 
of  the  design,  administration,  and  analysis  of  a  written  survey.  This 
survey  was  mailed  to  all  individuals  or  Air  Force  offices  within  the 
previously  determined  user  community.  Follow-up  letters  with  additional 
surveys  were  mailed  to  those  who  had  not  responded  to  the  initial 
mailing  within  three  weeks. 

A.  SURVEY  DESIGN 

The  WSSC  user  survey  was  designed  to  achieve  four  major  objectives: 

(1)  determine  who  among  the  respondents  were  current  or  potential  users, 

(2)  determine  the  level  of  satisfaction  that  existed  among  actual  users 
of  the  system,  (3)  determine  the  value  of  suggestions  previously  obtained 
from  users  for  changes  to  improve  the  system,  and  (4)  collect  additional 
suggestions  for  system  improvements. 

The  survey  consisted  of  a  total  of  twenty-six  questions  of  three 
types:  multiple-choice,  rating-scale,  and  open-ended.  For  the  sake  of 
brevity  and  ease  of  response,  most  questions  were  of  the  first  two 
types.  In  addition,  several  filter  questions  (questions  which  separate 
respondents  according  to  some  characteristic)  were  used  to  direct  the 
different  possible  types  of  respondents  to  appropriate  questions  or 
groups  of  questions.  Background  questions  requested  the  respondent's 
name  (optional),  type  of  employer,  and  work-related  tasks.  Respondents 
were  asked  if  they  were  familiar  with  the  WSSC  system.  Those  not 


familiar  with  the  system  were  directed  to  a  series  of  questions  designed 
to  classify  them  as  potential  users  or  not.  The  rest  of  the  survey 
consisted  of  questions  for  those  familiar  with  the  WSSC  system.  These 
questions  dealt  with  such  subjects  as  the  quality  of  the  data,  tasks 
for  which  the  data  was  used,  the  quality  of  the  system  as  a  whole,  and 
the  understandability  of  the  user's  manual.  In  addition,  respondents 
were  asked  to  rate  previously  obtained  suggestions  for  system  improve¬ 
ments  and  to  offer  additional  suggestions  for  enchancements  to  the  sys¬ 
tem.  All  respondents  were  asked  to  comment  on  the  survey  itself.  A 
copy  of  the  survey  is  in  Appendix  B. 

B.  SURVEY  POPULATION 

Surveys  were  sent  in  March  1984  to  all  individuals/offices  that 
Desmatics  had  identified  as  being  members  of  the  WSSC  user  community. 
Surveys  were  also  sent  to  those  offices  which  did  not  respond  to  the 
original  request  for  names  of  users/potential  users.  Multiple  copies 
of  the  survey  were  sent  to  those  offices  or  individuals  judged  likely 
to  be  aware  of  additional  users  to  whom  they  could  give  a  copy.  As 
mentioned  previously,  follow-up  letters  and  surveys  were  mailed  to 

e  who  did  not  return  a  completed  survey  within  three  weeks  of  the 
inic  '  mailing.  In  all,  a  total  of  782  surveys  were  mailed  to  484 
difft  snt  offices  or  individuals.  Of  the  484  offices  or  individuals, 

308  retu*ned  at  least  one  completed  survey.  Of  the  782  surveys  mailed, 
330  were  completed  and  returned. 


************************************************************* 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


Question  6.  Have  you  ever  seen  WSSC  output  or  used  WSSC 
data  in  your  work? 


66%  Yes 


34%  No 


************************************************************* 


Only  respondents  who  answered  yes  to  question  4  (N=156)  were 
instructed  to  answer  question  6.  There  were  no  nonresponses  to  this 
question.  Of  the  156  respondents,  103  (66%)  answered  yes  and 
53  (34%)  answered  no.  This  question  was  used  as  a  filter  question  to 
direct  respondents  to  question  7  (if  they  answered  yes)  or  to  question 
14  (if  they  answered  no).  As  noted  in  subsection  A,  the  respondents  who 
answered  no  to  this  question  comprise  the  HAS  NOT  SEEN  WSSC  OUTPUT  group. 
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50% 


Figure  2.  Responses  to  Question  5,  "How  familiar 
are  you  with  the  WSSC  system?" 


Circle 


A****************************************  AA AAAA AAAA AAAA AAAA  ********** 
* 

*  Question  5.  How  familiar  are  you  with  the  WSSC  system? 

J  the  number  on  the  scale  which  best  represents  your  opinion. 

* 

A 

J  Slightly  Familiar  12345  Very  Familiar 

A 

* 

AAAAAAAAAAAA AAAA AA AA AA AA AA AA AAAA AA AA AA AAAA AA AA AAAAAA AAAAAAAA AAAA AA AAA 


Only  respondents  who  answered  yes  to  question  4  (N=156)  were  in¬ 
structed  to  answer  question  5.  There  were  no  nonresponses  to  this 


question.  The  results  were  as  follows: 

Respondent  Group 

Number  of 
Respondents 

Mean 

Response 

J. 

Response 

2  3  4 

5 

WSSC  USERS 

55 

2.69 

16% 

24% 

40% 

15% 

5% 

HAS  SEEN  BUT  NOT  USED 

WSSC  OUTPUT 

48 

1.83 

42% 

40% 

14% 

2% 

2% 

HAS  NOT  SEEN  WSSC  OUTPUT 

53 

1.79 

49% 

26% 

21% 

4% 

0% 

OVERALL 

156 

2.12 

35% 

29% 

26% 

7% 

3% 

Not  surprisingly,  WSSC  USERS  indicated  the  highest  degree  of  famil¬ 
iarity  with  the  WSSC  system.  In  addition.  Figure  2  shows  that  DOD  con¬ 
tractor  WSSC  Users  considered  themselves  much  more  familiar  with  WSSC 
than  did  AF  WSSC  Users. 
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■***  ********  * : 


***********: 


******************************************************************* 

* 
* 
* 


jH* 

Question  4.  Are  you  familiar  with  the  Air  Force  WSSC  system?  * 

J. 

53%  Yes  47%  No 


******************************************* ************************ 


All  respondents  were  instructed  to  answer  question  4.  There  were 


no  nonresponses  to  this  question.  Of  the  293  respondents,  156  (53%) 


Force  or  by  a  DOD  contractor.  Table  4  provides  corresponding  data  for 
these  two  groups.  An  examination  of  the  table  shows  that  the  tasks 
performed  by  the  largest  percentage  of  respondents  in  each  subgroup 
were : 


AF 

WSSC  Users 

DOD 

Contractor  WSSC  Users 

1. 

Systems  Comparison  (58%) 

1. 

Trade-Off  Analysis  (88%) 

2. 

Logistics  Forecasting/Management 
(58%) 

2. 

Life  Cycle  Cost  Modeling/Fore¬ 
casting  (88%) 

3. 

Life  Cycle  Cost  Modeling/Fore¬ 
casting  (55%) 

3. 

Life  Cycle  Cost  Management 
(61%) 

4. 

Reliability /Maintainability 
Studies  (42%) 

4. 

Reliability /Maintainability 
Studies  (61%) 
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Air  Force 

DOD  Contractor 

WSSC  Users 

WSSC  Users 

Task 

N=33 

N=18 

B. 

LIFE  CYCLE  MANAGEMENT 

39% 

61% 

C. 

TRADE-OFF  ANALYSIS 

39% 

88% 

D. 

BUDGET  PREPARATION 

27% 

22% 

E. 

LIFE  CYCLE  COST 
MODELING/FORECASTING 

55% 

88% 

F. 

RELIABILITY /MAINTAINABILITY 
STUDIES 

42% 

61% 

G. 

LOGISTICS 

FORECASTING/MANAGEMENT 

58% 

39% 

H. 

MANPOWER 

FORECASTING/MANAGEMENT 

12% 

28% 

I. 

SYSTEMS  COMPARISON 

58% 

56% 

J. 

DSARC  SUBMISSIONS 

12% 

16% 

K. 

EVALUATE  PRODUCT  PERFORMANCE 
AGREEMENTS 

12% 

22% 

L. 

WAR  READINESS  ASSESSMENTS 

24% 

28% 

M. 

POM  SUBMISSIONS 

36% 

6% 

N. 

SUPPORTABILITY  ANALYSIS 

33% 

44% 

0. 

READINESS/SUSTAINABILITY 

ANALYSIS 

36% 

39% 

P. 

OTHER 

12% 

5% 

Table  4.  Summary  of  Responses  to  Question  3  for 
AF  and  DOD  Contractor  WSSC  Users. 
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*********************************************************************** 
*  * 

*  * 

*  * 

*  Question  3.  Which  of  the  following  tasks  involving  aircraft  * 

*  do  you  perform  in  your  work?  Please  circle  all  that  apply.  * 

*  * 

*  ft 

*  7 L  A.  My  work  does  not  involve  aircraft  * 

*  30%  B.  Life  cycle  cost  management  * 

*  35%  C.  Trade-off  analysis  * 

*  27%  D.  Budget  preparation  J 

*  40%  E.  Life  cycle  cost  modeling/forecasting  * 

*  35%  F.  Reliability/maintainability  studies  * 

*  40%  G.  Logistics  forecasting/management  * 

*  18%  H.  Manpower  forecasting/management  * 

*  34%  I.  Systems  comparison  (existing  or  conceptual)  * 

*  9%  J.  DSARC  submissions  * 

*  14%  K.  Evaluate  product  performance  agreements  (warranties,  * 

*  maintenance  agreements,  etc.)  * 

*  18%  L.  War  readiness  assessments  * 

*  32%  M.  POM  submissions  * 

JL 

*  32%  N.  Supportability  analysis  * 

*  28%  0.  Readiness/sustainability  analysis  * 

$  15%  P.  Other(s),  please  specify:  * 

5  * 

*  * 
*********************************************************************** 


All  respondents  were  instructed  to  answer  question  3.  There  were 
no  nonresponses  to  this  question.  As  can  be  seen  from  the  results  above, 
the  tasks  performed  by  the  largest  percentage  of  respondents  were  Life 
Cycle  Cost  Modeling/Forecasting  (40%),  Logistics  Forecasting/Management 
(40%),  Reliability/Maintainability  Studies  (35%),  and  Trade-off  Analysis 
(35%). 

Table  3  provides  a  breakdown  of  the  responses  by  respondent  group. 

As  can  be  seen  from  this  table,  the  tasks  performed  by  at  least  40%  of 
WSSC  USERS  were  Life  Cycle  Cost  Modeling/Forecasting  (69%),  Trade-Off 
Analysis  (56%),  Systems  Comparison  (55%),  Logistics  Forecasting/Management 
(47%),  Reliability /Maintainability  Studies  (47%),  and  Life  Cycle  Cost 
Management  (47%). 

Because  the  vast  majority  of  WSSC  USERS  are  employed  by  the  Air 


Question  2.  Who  is  your  current  employer? 


************************************************************************** 
* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

************************************************************************** 


86%  A. 
0%  B. 
1%  C. 


Air  Force 

Army 

Navy 


1%  D. 
10%  E. 
2%  F. 


Department  of  Defense 
Department  of  Defense  Contractor 
Other,  please  specify 


All  respondents  were  instructed  to  answer  question  2.  Of  the  293 
respondents,  there  were  six  nonresponses  to  this  question.  A  look  at  the 
results  above  shows  that  the  Air  Force  and  DOD  contractors  accounted  for 
96%  of  the  respondents.  A  breakdown  of  employer  by  respondent  group  is 
illustrated  in  Figure  1.  An  examination  of  this  figure  shows  that  33% 
of  WSSC  USERS  were  DOD  contractors  and  61%  were  AF  employees.  In  con¬ 
trast,  the  other  four  respondent  groups  consisted  almost  exclusively  of 
AF  employees.  (Note  that  Desmatics  uses  the  term  "Air  Force  employees" 
to  include  both  military  and  civilian  personnel.) 
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B.  ANALYSIS  OF  QUESTIONS 


The  following  pages  provide  a  summary  and  analysis  of  the  responses 
to  each  survey  question.  Note  that  question  1  is  not  discussed  because 
it  was  an  optional  question  requesting  the  respondent's  name  and  address. 


-11- 


would  be  useful  In  their  work  (Q23-yes) .  Finally,  respondents  were 
classified  into  the  UNFAMILIAR  WITH  WSSC;  NOT  POTENTIAL  USER  group 
if  they  were  not  familiar  with  WSSC  (Q4-no)  and  felt  that  such  a  system 
would  not  be  useful  in  their  work  (Q23-no).  Surveys  in  which  the  respon¬ 
dent  did  not  answer  enough  questions  to  be  classified  or  gave  conflicting 
answers  to  questions  used  for  classification  (e.g.,  answering  Q4-no  and 
Q6-yes)  were  classified  as  invalid  and  excluded  from  analysis. 

Of  the  330  surveys  returned,  37  were  classified  as  invalid.  A 
breakdown  of  the  remaining  293  surveys  according  to  respondent  group 
is  given  in  Table  2. 


Table  2.  Breakdown  by  Group 


Respondent  Group 


Number  of  Percentage  of 
Respondents  Respondents 


Familiar: 

WSSC  USER  55  19% 

HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  48  16% 

HAS  NOT  SEEN  WSSC  OUTPUT  53  18% 

Not  Familiar: 

UNFAMILIAR  WITH  WSSC:  POTENTIAL 
USER  74  25% 

UNFAMILIAR  WITH  WSSC:  NOT 

POTENTIAL  USER  _63  22% 

293  100% 


-10- 


( 


t 


The  criteria  for  classifying  respondents  into  the  five  groups  are 
shown  in  Table  1.  Respondents  were  classified  into  the  WSSC  USER  group 
if  they  were  familiar  with  the  WSSC  system  (Q4-yes)  and  had  used  WSSC 
for  one  or  more  tasks  (circled  one  or  more  of  Q8B-Q8P) .  Respondents 
were  classified  into  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  group  if 


they  were  familiar  with  WSSC  (Q4-yes)  and  had  seen  WSSC  output  (Q6-yes) , 
but  had  never  used  WSSC  data  (circled  Q8A) .  Respondents  were  classified 
into  the  HAS  NOT  SEEN  WSSC  OUTPUT  group  if  they  were  familiar  with  the 
WSSC  system  (Q4-yes)  and  had  not  seen  or  used  WSSC  data  (Q6-no) . 


Table  1.  Respondent  Group  Classification 


Group 

Familiar : 

24 

22 

22 

223 

V 

WSSC  USER 

Yes 

Yes 

BfCy • » • )Or  P 

- 

ft 

HAS  SEEN  BUT  NOT 

USED  WSSC  OUTPUT 

Yes 

Yes 

A 

_ 

1 

HAS  NOT  SEEN  WSSC 
OUTPUT 

Yes 

No 

_ 

Not  Familiar: 

•- 

— 

UNFAMILIAR  WITH  WSSC: 
POTENTIAL  USER 

No 

_ 

_ 

Yes 

ft 

UNFAMILIAR  WITH  WSSC: 
NOT  POTENTIAL  USER 

No 

- 

- 

No, 

Don't 

Know 

Respondents  were  classified 

into 

the  UNFAMILIAR 

WITH 

WSSC: 

fe 

POTENTIAL  USER  group  if  they 

were 

not 

familiar  with  WSSC 

(Q4-no) 

but 

felt  that  a  system  which  reports  historical  O&S  costs  for  AF  aircraft 


I 


IV.  RESULTS  AND  ANALYSIS 


This  section  is  divided  into  two  subsections.  In  subsection  A 
the  respondents  are  classified  into  two  main  groups  according  to 
whether  or  not  they  are  familiar  with  the  WSSC  system.  The  respondents 
who  are  familiar  with  WSSC  are  further  categorized  based  on  their  amount 
of  experience  with  WSSC.  The  respondents  unfamiliar  with  WSSC  are  fur¬ 
ther  categorized  based  on  whether  they  feel  WSSC  would  be  useful  in 
their  work.  In  subsection  B  the  responses  of  each  survey  question  are 
summarized  and  analyzed.  Conclusions  and  recommendations  are  presented 
in  Section  V. 


A.  CLASSIFICATION  OF  RESPONDENTS 


Based  on  the  responses  to  Questions  4,  6,  8,  and  23,  each  respon¬ 
dent  was  classified  into  one  of  five  mutually  exclusive  groups.  These 


1)  Familiar  with  WSSC:  WSSC  USER 

2)  Familiar  with  WSSC:  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT 

3)  Familiar  with  WSSC:  HAS  NOT  SEEN  WSSC  OUTPUT 

4)  UNFAMILIAR  WITH  WSSC:  POTENTIAL  USER 

5)  UNFAMILIAR  WITH  WSSC:  NOT  POTENTIAL  USER. 

In  the  succeeding  discussion  each  group  will  be  referred  to  only  by 
the  underlined  portion  of  the  classification  description  given  above. 

In  addition,  the  shorthand  notation  Q2,  Q3,  etc.  will  be  used  to  denote 


questions  2,  3,  etc. 
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********************************************************** 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


Question  7.  How  understandable  is  the  WSSC  output? 


Very  Difficult 
to  Understand 


Very  Easy 
to  Understand 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


********************************************************** 


Only  respondents  who  answered  yes  to  question  6  (N=103)  were 
instructed  to  answer  question  7.  There  were  five  nonresponses  to 
this  question.  The  results  were  as  follows: 


Number  of 
Respondents 

Mean 

Response 

l 

Response 
2  3 

4 

5 

WSSC  USERS 

52 

3.37 

21 

12% 

45% 

29% 

12% 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

46 

3.13 

21 

15% 

59% 

15% 

9% 

OVERALL 

98 

3.26 

21 

13% 

52% 

23% 

10% 

As  can  be  seen  from  the  results  above,  WSSC  USERS  rated  WSSC  output 
as  being  slightly  more  understandable  than  did  respondents  in  the  HAS  SEEN 
BUT  NOT  USED  WSSC  OUTPUT  group.  As  Figure  3  indicates,  within  the  WSSC 
USERS  group,  DOD  contractors  found  WSSC  more  understandable  than  did  AF 
employees. 
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Question  8.  Please  circle  any  of  the  following  tasks  for  which 
you  have  used  WSSC  data. 


********************************************************************** 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

s 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

********************************************************************* 


47%  A. 
29%  B. 
35%  C. 
4%  D. 
49%  E. 
24%  F. 
29%  G. 

7%  H. 
33%  I. 
7%  J. 
2%  K. 

9%  L. 
7%  M. 
13%  N. 
11%  0. 
11%  P. 


I  have  not  used  WSSC  data  (go  to  question  10) 

Life  cycle  cost  management 
Trade-off  analysis 
Budget  preparation 

Life  cycle  cost  modeling/forecasting 
Reliability /maintainability  studies 
Logistics  forecasting/management 
Manpower  forecasting/management 
Systems  comparison  (existing  or  conceptual) 

DSARC  submissions 

Evaluate  product  performance  agreements  (warranties, 
maintenance  agreements,  etc.) 

War  readiness  assessments 
POM  submissions 
Supportability  analysis 
Readiness /sustainability  analysis 
Other (s),  please  specify: 


Only  respondents  who  answered  yes  to  question  6  (N=103)  were  in¬ 
structed  to  answer  question  8.  There  were  no  nonresponses  to  this  ques¬ 
tion.  Of  the  103  respondents,  48  (47%)  indicated  they  had  not  used  WSSC 
data  (i.e.,  circled  Q8A) ,  while  55  (53%)  indicated  they  had  used  WSSC 
(i.e.,  circled  one  or  more  of  Q8B-Q8P) .  As  noted  in  subsection  A, 
respondents  in  the  former  group  comprise  the  HAS  SEEN  BUT  NOT  USED  WSSC 
OUTPUT  group;  respondents  in  the  latter  group  comprise  the  WSSC  USERS 
group. 

Table  5  shows  the  percentage  of  WSSC  USERS  who  use  WSSC  to  per¬ 
form  a  given  task.  A  breakdown  of  WSSC  USERS  into  Air  Force  and  DOD 
contractors  is  also  presented.  In  addition.  Table  5  reports  the  ratio 
(R)  of  the  number  of  WSSC  USERS  who  circled  a  particular  task  in  ques¬ 
tion  8  to  the  number  of  WSSC  USERS  who  circled  that  task  in  question  3. 


-24- 


»*»»»»*»»»»»  »»*»*-»»»»»»»»»»»»  ****#■  a-***-***  »»*» 


All  WSSC  Users  (N-55)  WSSC  Users;  Air  Force  (H-33)  WSSC  Users:  POP  Contractors  (N-18 
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Table  5.  Summary  of  Responses  to  Question  8  by  WSSC  USERS 


Of  the  WSSC  USERS  who  perform  a  given  task,  this  ratio  represents  the 
percentage  who  use  WSSC  data  in  performing  that  task. 


As  indicated  by  Table  5,  among  WSSC  USERS  the  major  uses  for  WSSC 
data  were  Life  Cycle  Cost  Modeling/Forecasting  (49%),  Trade-off  Analysis 
(35%),  and  Systems  Comparison  (33%).  Among  AF  WSSC  USERS  the  major  uses 
were  Life  Cycle  Cost  Modeling/Forecasting  (39%),  Systems  Comparison  (33%), 
and  Life  Cycle  Cost  Management  (21%).  Finally,  among  DOD  contractor 
WSSC  USERS  the  major  uses  for  WSSC  data  were  Trade-off  Analysis  (67%) 
and  Life  Cycle  Cost  Modeling/Forecasting  (61%). 

Two  additional  observations  can  be  made  based  on  an  inspection  of 
Table  5.  The  first  of  these  is  that  the  percentage  of  DOD  contractors 
who  use  WSSC  to  perform  each  of  the  tasks  listed  in  Table  5  is  in  general 
larger  than  the  corresponding  percentage  of  AF  employees  who  use  WSSC 
for  these  tasks.  The  second  is  that  DOD  contractors  are  more  likely  to 
use  WSSC  data  to  perform  a  given  task  than  are  AF  employees. 
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******************************************************** 
*  * 

*  * 

J  Question  9.  How  useful  is  WSSC  data  in  your  work?  J 

£  5 

J  Not  Useful  12345  Very  Useful  J 

*  i 

*  * 

******************************************************** 


Only  WSSC  USERS  (N»55)  were  instructed  to  answer  question  9. 
There  was  one  nonresponse  to  this  question.  The  responses  are  sum¬ 
marized  below: 


WSSC  USERS 


Number  of  Mean 

Respondents  Response 


Response 

1  2  3  4  5 


54 


3.26 


7%  15%  35%  30%  13% 


As  can  be  seen  from  the  results  above,  WSSC  USERS  in  general  rated 
WSSC  data  as  being  useful  in  their  work.  Furthermore,  as  indicated  by 
the  results  below,  DOD  contractor  WSSC  USERS  rated  WSSC  data  much  more 
useful  in  their  work  than  did  Air  Force  WSSC  USERS. 


Number  of 

Mean 

Response 

WSSC  USERS: 

Respondents 

Response 

\  2  3  4 

J5 

Air  Force 

32 

2.88 

13%  25%  28%  31% 

3% 

DOD  Contractors 


18 


3.83 


0%  0%  44%  28%  28% 


I'  »J  ».  ■  .  MU'.  I""  »■  ■  I  l.  H.M.HL  »  Jl 


9 


***************************************************************** 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


Question  10.  Do  you  have  a  need  for  aircraft  operating  and 
support  cost  data  reported  at  any  of  the  following  levels? 
Please  circle  all  that  apply. 


54% 

A. 

Worldwide 

54% 

B. 

Command 

44% 

C. 

Base 

38% 

D. 

Squadron 

75% 

E. 

Mission  Design  Series 

28% 

F. 

Mission  Design 

16% 

G. 

Mission 

4% 

H. 

Don't  know 

10% 

I. 

Other 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


* 

* 

* 

* 

* 

* 

* 

* 


***************************************************************** 


WSSC  USERS  (N-55) 

and  the  HAS  SEEN  BUT  NOT 

USED  WSSC  OUTPUT  group 

(N=48)  were  instructed 

to  answer 

question  10. 

There  were  two  nonresponses 

■:  - 

to  this  question.  The 

responses 

are  summarized 

below: 

HAS  SEEN  BUT  NOT 

ALL 

WSSC  USERS 

USED  WSSC  OUTPUT 

Level 

(N=101) 

(N-55) 

(N=46) 

...  , 

WORLDWIDE 

54% 

71% 

35% 

COMMAND 

54% 

62% 

46% 

•  '< 

BASE 

44% 

45% 

41% 

- 

SQUADRON 

38% 

40% 

35% 

MISSION  DESIGN  SERIES 

75% 

80% 

70% 

MISSION  DESIGN 

28% 

33% 

22% 

■'* 

MISSION 

16% 

20% 

11% 

DON'T  KNOW 

4% 

2% 

7% 

.  - 

OTHER 

10% 

4% 

17% 

_ :  ■ 

Although  respondents  indicated  a  need  for  cost  data  at  all  these  levels 
the  Mission  Design  Series  level  was  chosen  most  often. 


*************************************************************** 


* 

* 

* 

5 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


Question  11.  The  following  suggestions  have  been  made  for 
additions  or  changes  to  the  WSSC  system.  Please  rate  the 
usefulness  of  these  suggestions  as  follows: 

O-Not  Useful  1-Somewhat  Useful  2-Very  Useful 
3-Essential  4-Don' t  Know 

(See  Table  6  for  the  list  of  suggestions) 


*************************************************************** 


WSSC  USERS  (N*55)  and  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  group 
(N=48)  were  instructed  to  answer  question  11.  The  number  of  nonresponses 
differed  from  suggestion  to  suggestion.  The  responses  are  summarized 
for  both  groups  in  Table  6.  Figure  4  provides  a  graph  of  the  mean  ratings 
of  the  suggestions  and  their  associated  95%  confidence  intervals.  In¬ 
spection  of  Figure  4  reveals  that,  based  on  the  mean  ratings,  the  sug¬ 
gestions  appear  to  fall  into  four  groups,  ranging  from  most  to  least  use¬ 
ful.  These  groups  are: 

Group  I 

K:  Provide  cost  categories  which  correspond  to  the  CORE  factors  (AFR  173-13) 

J:  Include  an  optional  brief  description  of  what  costs  are  included  in 

each  category  with  cost  reports. 

Group  2 

H:  Provide  separate  visibility  of  software  maintenance  costs. 


C:  Add  capability  to  get  data  for  two  or  more  years  in  the  same 
specified  constant  year  dollars. 


Nuabcr  of  Don't  Not  Sonewhat  Very  Mean 

SUGGESTION  Respondent a  Know  Useful  Useful  Useful  Essential  Response 
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Group  3 

G:  Expand  system  to  include  acquisition  costs. 

B:  Add  an  optional  narrative  description  of  aircraft  history  and 
major  changes. 

A:  Change  base  level  maintenance  cost  categories  from  organizational 
categories  to  functional  categories. 

F:  Include  an  optional  cumulative  description  of  all  changes  to 
WSSC  system. 

I:  Add  costs  for  Air  Force  Reserve  and  Air  National  Guard  aircraft. 

D:  Include  reports  on  strategic  missiles. 

Group  4 

E:  Include  reports  on  spacecraft/space  systems. 

It  is  worth  noting  that  all  proposed  changes  had  a  mean  rating 
greater  than  1.0,  indicating  that  on  the  average  all  suggestions  proposed 
in  the  survey  were  considered  at  least  somewhat  useful  by  the  respondents 
Many  respondents  provided  their  own  suggestions.  A  complete  unedited 
listing  of  these  suggestions  follows: 

1.  We  utilize  the  CORE  model  so  I  would  be  interested  in  having 
CORE  and  WSSC  system  very  compatible. 

2.  Include  the  LSC  model  to  compare  to  WSSC  173-13 

3.  Collect  costs  that  track  to  budget  categories  (EEIC,BP,etc.) 

4.  Separate  and  breakout  modification  installation  costs  in  depot 
level  -eports. 


Provide  pipeline  spares  costs  as  well  as  condemnations. 


5.  Give  all  limitations  and  constraints  of  the  costs  from  inputs 
or  product  outputs  and  tell  user  the  impact  and  what  it  means 
for  applicability  to  cost  studies. 

Add  capability  to  generate  reports  to  engine  TMS-engine  costs 
are  getting  so  much  visibility  we  need  to  get  products  that  can 
isolate  costs  to  specific  engine. 

6.  Segregate  simulator  costs  from  MDS  data;  i.e.,  Repl  spares,  depot 
mx,  etc. 

Segregation  of  ASIF  funded  and  O&S  funded  operations 

7.  Costs  by  Program  Element  Code  (PEC) 

Costs  by  Element  of  Expense  Investment  Code  (EEIC)  and  budget 
program 

8.  Why  not  by  squadron  FH  per  PAA  extra 


9.  Cost  categories  for  Work  Unit  Codes 


10.  Show  two-digit  Work  Unit  Code  costs;  also  break  out  costs  by 
labor  hours  and  material  dollars 


11.  On  sustaining  investment  -  define  what  is  included  in  each  element 
of  investment.  Same  comment  for  MATERIEL. 

Define  source  data  for  each  type  input. 

Provide  user's  guide. 

12.  Re-format  IAW  attachment.  [The  respondent  attached  an  example  of 
WSSC  data  which  had  been  reformated.  The  format  contained  the  same 
cost  categories  as  WSSC  CAIG  Format,  but  presented  several  MDSs 

on  one  page . ] 


13.  Incorporate  my  suggestions  submitted  12  Feb  1982  Control  No.  82.0077 
design  problem  report  (DPR)  relative  to  H036C  (WSCRS) 

Make  corrections  to  D041  condemnation  deficiences 

Utilize  the  USAF/Maint.  Cost  System  (MCS)  by  USAF  Finance  &  Accounting 
Center,  Denver  CO  to  report  base  maint.  support  cosf  by  MDS/TMS  by  PEC. 


14.  Provide  the  data  in  magnetic  tape  format 

15.  Release  data  as  frequently  as  possible;  do  not  hold  for  changes 
in  format 

16.  Separate  contract  logistic  support  cost  breakouts,  including  leased 
aircraft 

FMS  identifiable  items  broken  out  by  MDS 
Interservice  costs  broken  out  by  MDS 

17.  Include  reports  on  simulators 

18.  Identify  direct  and  overhead  rates  and  where  applied 


********************************************************************* 
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* 

* 

* 

* 

* 

* 

* 

* 

* 

* 
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* 

* 

* 

* 

* 

* 

* 
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* 

* 

* 

* 
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Question  12.  Please  rate  the  accuracy  of  the  costs  produced 
by  the  WSSC  system  for  each  of  the  following  cost  categories. 


0-Don' t  know 


1-Too  Low 


2-About  Right 


3- 

Too  High  4-Sometimes  Too 

High 

,  Sometimes  Too  Low 

A. 

Unit  Operations 

F. 

General  Depot  Support 

B. 

Below  Depot  Maintenance 

G. 

Depot  Installation  Support 

C. 

Installation  Support 

H. 

Medical  Care 

D. 

E. 

Sustaining  Investment 

Depot  Maintenance 

I. 

Permanent  Change  of  Station 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


********************************************************************* 


WSSC  USERS  (N=55)  and  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  group 
(N=48)  were  instructed  to  answer  question  12.  The  number  of  nonresponses 
differed  from  category  to  category.  The  responses  are  summarized  in 
Table  7.  It  can  readily  be  seen  from  this  table  that  a  very  large  per¬ 
centage  of  respondents  indicated  that  they  "don't  know"  the  accuracy  of 
costs  provided  by  the  various  cost  categories.  Among  the  few  respon¬ 
dents  who  rated  the  accuracy  (i.e.,  responded  other  than  don't  know) 
of  the  various  categories  the  majority  felt  that  all  categories  except 
for  Installation  Support,  Medical  Care,  and  Permanent  Change  of  Station 
produced  inaccurate  costs  (i.e.,  either  too  low,  too  high,  sometimes  too 
low,  sometimes  too  high). 

Some  respondents  provided  a  reason  for  their  accuracy  rating.  A 
complete  unedited  listing  of  these  reasons  for  each  category  is  provided. 
It  should  be  noted  that  some  respondents  gave  the  same  reason  for  several 
categories . 
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***************************************************************** 
* 

* 

* 

* 

* 

* 


Question  16.  How  understandable  is  the  WSSC  users  manual? 


Very  Hard 
to  Understand 


Very  Easy 
to  Understand 


Question  17.  How  useful  is  the  WSSC  manual? 


Not  Useful  1 


5  Very  Useful 


* 

* 

* 

* 

* 

* 
k 
k 
* 

* 

* 

* 

* 

* 

* 

* 

* 
k 
* 

* 

•k 
k 
k 
k 
k 
* 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
k 
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Question  18.  How  would  you  rate  the  level  of  detail  in  the 
WSSC  users  manual? 


Not  Enough 
Detail 


Too  Much 
Detail 


Question  19.  How  easy  is  the  WSSC  users  manual  to  use? 


Very  Easy  1 


5  Very  Difficult 


* 

* 

* 

k 

k 

k 

k 

k 

k 

*.V 

* 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 
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k 

k 
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k 

k 

k 

k 

k 

k 

k 

k 
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k 
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k 

k 

k 
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k 

k 

k 
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Only  respondents  who  answered  yes  to  question  15  (N=61)  were  in¬ 
structed  to  answer  questions  16  through  19.  There  was  one  nonresponse 
to  question  17  and  one  nonresponse  to  question  19.  There  were  no  non¬ 
responses  to  questions  16  and  18.  The  results  are  summarized  below: 


Q 1 6 


Group 

Number  of 
Respondents 

Mean 

Response 

Response 
2  3 

'  4 

5 

ALL  USERS 

33 

3.00 

3% 

30% 

30% 

36% 

0% 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

18 

3.28 

6% 

11% 

39% 

39% 

6% 

HAS  NOT  SEEN  WSSC 
OUTPUT 

10 

3.40 

0% 

10% 

60% 

10% 

20% 

OVERALL 

61 

3.15 

3% 

21% 

38% 

33% 

5% 

Number  of 

WSSC  USER: 

Respondents 

Yes 

Air  Force 

33 

39% 

DOD  Contractor 

18 

94% 

» 


I 


» 


I 


************************************************************* 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


Question  15.  Have  you  ever  read  or  used  the  WSSC  users 
manual  (AFR  400-31  Vol.  II)? 


40%  Yes 


60%  No 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


************************************************************* 


All  respondents  familiar  with  the  WSSC  system,  i.e.,  WSSC  USERS, 

HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT,  and  HAS  NOT  SEEN  WSSC  OUTPUT  groups, 
-  -  # 

(N-156)  were  instructed  to  answer  question  15.  There  were  three  non¬ 
responses  to  this  question.  The  responses  for  the  various  respondent 
groups  are  summarized  below: 


Group 

Number  of 
Respondents 

Yes 

• 

WSSC  USERS 

55 

60% 

HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT 

48 

38% 

HAS  NOT  SEEN  WSSC  OUTPUT 

50 

20% 

OVERALL 

153 

40% 

» 

A  look  at  the  results  above  shows  that  40%  of  the  respondents 
to  this  question  had  read  or  used  the  WSSC  users  manual.  In  addition, 
as  might  be  expected,  a  larger  percentage  of  WSSC  USERS  (60%)  had  read 
or  used  the  manual  than  those  in  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT 
(38%)  or  the  HAS  NOT  SEEN  WSSC  OUTPUT  (20%)  groups.  Furthermore,  as 
can  be  seen  from  the  results  below,  94%  of  DOD  contractor  WSSC  USERS 
had  read  or  used  the  manual,  while  only  39%  of  Air  Force  WSSC  USERS 
had  read  or  used  the  manual. 
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» 


25 .  Don ' t  know  how 


26.  Were  not  aware  that  WSSC  data  was  available. 

27.  Currently  trying  to  determine  which  data  we  may  be  able  to  use. 

28.  Just  beginning-really  need  this  system  for  tanker  (KC-135) 
program  decision  alternatives 

29.  Just  learning  about  their  data 

30.  The  short  time  periods,  and  lack  of  summaries  over  time  make  it 
too  hard  to  format  for  my  purposes 

31.  Have  a  better  data  base  (WSCRS);  my  data  base  develops  the  cost 
history  in  a  form  I  can  use;  WSSC  does  not 

32.  The  mandates  of  current  budgeting  for  logistics  directives  are  to 
develop  valid  costing  techniques  for  existing  (inventory)  systems 
and  proposed  (acquisition)  systems  through  the  POM  (Program 
Objective  Memorandum)  years.  Historical  data  may  be  valid  for 
projecting  out-year  and  near  term  costs  of  support  for  inventory 
systems,  but  in  view  of  rapidly  evolving  technology  this  data 
proration  or  factoring  cannot  be  valid  for  new  systems.  This  is 
especially  true  in  the  C-E  area. 

33.  Recently  just  inquired  for  information,  hopefully  some  of  the  data 
will  be  helpful  and  useful  in  present  job  assignment.  The  base 
works  three  aircraft,  F15,  C130,  C141,  performing  both  PDM  and 
TCTO  updates. 

34.  WSSC  data  seems  to  be  of  a  general  nature,  applicable  to  complete 
weapon  systems  or  fleets.  We  at  this  level  are  concerned  with 
reliability  of  individual  components  of  the  weapon  systems.  Parts 
failures,  etc. 


5.  Data  is  normally  used  in  our  area  by  the  analysis  folks.  We 
only  see  the  results. 

6.  Awaiting  data,  just  got  on  listing. 

7.  Used  by  members  of  my  group  who  perform  LCC  analysis 

8.  I  have  recently  changed  positions  and  no  longer  support  aircraft 
directly. 

9.  Not  on  distribution. 

10.  Data  is  not  available  to  me. 

11.  Have  not  required  as  yet. 

12.  To  my  knowledge  the  data  has  not  been  made  available  to  our 
organization. 

13.  I  am  awaiting  my  first  request  for  output. 

14.  I  am  a  supervisor  and  would  not  personally  use  the  data  but  my 
people  would. 

15.  Have  only  received  and  started  to  use  CSCS  data.  WSSC  data  was 
not  specified  in  company's  original  data  request. 

16.  Not  aware  of  what's  available. 

17.  Opportunity  has  not  been  right  to  use  in  place  of  other  data. 

It  will  eventually  be  used. 

18.  Have  been  suing  the  CORE  model  for  O&S  cost  estimating.  Felt 
that  your  system  needed  a  few  years  of  data  gathering  to  obtain 
reliable  estimates. 

19.  I  personally  have  not.  The  F— 1 5  DPMI  office  has,  and  has  completed 
a  questionnaire.  The  Maverick  DPML  has  a  unique  LCC  model  and  has 
expressed  little  interest  in  VAMOSC. 

20.  Cost  analysts  have  usually  supported  us. 

21.  Requested  WSSC  data  on  a  couple  of  occasions,  but  never  received 
it 

22.  Have  not  had  the  opportunity  to  use  data 

23.  Lack  of  confidence  in  data  by  users  and  OPRs. 

24.  TASC  maintains  its  own  data  accumulation  system  called  TOPAS. 

TASC  is  also  the  prime  contractor  for  the  WISMIS  system. 


19. 

I  have  only  recently  recieved  a  sample  of  the  output  available. 

I  may  begin  using  it  to  crosscheck  173-13  cost  factor  reason- 
ability. 

» 

1 

20. 

We  were  told  that  engine  data  was  bad. 

•y-v- 

21. 

Probably  will  in  the  future,  but  for  now  existing  data  bases 
have  more  history. 

• 

22. 

Responsible  for  tasks  including  VAMOSC  data  analysis;  however 
lack  of  manpower  has  resulted  in  little  usage  of  WSSC  data. 

23. 

Study  data  requirements  did  not  warrant 

24. 

VAMOSC  WSSC  data  is  not  available  for  ICBM  systems 

• 

25. 

I  have  requested  data  however  I  have  not  come  upon  a  situation 
where  the  data  is  useful 

26. 

Was  able  to  solve  problem  without  WSSC  data 

» 

27. 

Data  useful  but  not  essential  since  others  share  responsibility 
for  costing.  Not  very  familiar  with  products  and  procedures. 

Early  data  bases  were  incomplete-not  sure  of  current  status-and 
this  reduced  usefulness. 

.  - 

28. 

Job  requires  use  of  CSCS  data  more  often 

k 

29. 

Found  tremendous  differences  between  different  models  within 
same  MDS 

*  .  ■' 

30. 

Little  knowledge  of  system  until  recently. 

4*  ..w  W 

} 

HAS 

NOT  SEEN  WSSC  OUTPUT 

1. 

Do  not  know  format/content/availability/accessability 

> 

2. 

I  requested  and  received  info  only  on  the  B-52 .  This  is  our 
only  experience. 

3. 

Automatic  Test  Equipment  has  not  been  added  in  this  model-when 
it  is  I  will  have  a  definite  need  for  this  WSSC  output. 

» 

4. 

Situation  has  not  arisen  where  this  data  would  be  more  beneficial 
than  that  obtained  directly  from  the  WSCRS  system. 
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1 

‘  V  '  '  * 

1 

HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT 


1.  This  data  not  used  in  my  studies/analysis 

2.  Haven't  started  yet  -  but  will  in  the  future. 

3.  HQ  AFLC  requested  potential  user  of  Weapon  System  Support 
Development  (WSSD) .  I  requested  additional  information  -  that 
is  how  I  was  introduced  to  both  programs  -  have  not  used  either. 

4.  Lack  of  quick  understanding  of  cost  breakdowns. 

Lack  of  time  to  study  the  data  in  depth  for  better  understanding. 

5.  Staff  function  and  new  at  location.  Recently  transferred  from 
WPAFB  to  RAFB. 

6.  We  haven't  determined  if  they  are  useful  in  our  operation. 

7.  Previously  used  MCS-switching  to  VAMOSC  now. 

8.  Info  that  could  be  used  must  be  for  AF  Reserve.  I  have  not  had 
the  opportunity  to  utilize  the  WSSC  data  base  as  yet. 

9.  Looking  at  data  for  possible  use  but  haven't  reached  the  point 
where  decision  has  been  made  to  use  it. 

10.  Haven't  determined  the  usefulness  of  the  WSSC  data  for  my  job 

11.  Job  required  cost  data-but  needed  many  years  of  historic  costs 
by  weapon  system  and  was  able  to  get  necessary  info  from  another 
system 

12.  Although  the  job  does  not  necessarily  require  this  data,  with  a 
greater  understanding  of  the  system  it  may  become  useful  in  my  job 

13.  Just  offered  the  data  recently. 

14.  My  employees  use  it.  I  am  a  supervisor. 

15.  Depends  upon  study  under  development /review. 

16.  Have  had  no  recent  requirement  for  WSSC  data. 

17.  1.  Lack  of  knowledge  about  obtaining  data  base. 

2.  Unknown  acceptance  of  data  at  Air  Staff  level. 

3.  Data  should  be  recent. 


18.  General  availability  of  data  in  AFR  173-13  has  been  adequate 
except  for  Wpn  sys  trainers  (these  data  are  deplorable) 


********************************************************************* 

* 

* 

*  Question  14.  If  you  have  not  used  WSSC  data,  please  specify  why. 

* 

* 

*  23%  A.  Job  does  not  require  that  type  of  data. 

*  77%  B.  Other,  please  specify: 

* 

* 

********************************************************************* 


Respondents  in  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  (N=48)  and 
HAS  NOT  SEEN  WSSC  OUTPUT  (N=53)  groups  were  instructed  to  answer  ques¬ 
tion  14.  There  were  13  nonresponses  to  this  question.  The  responses 
are  summarized  below: 


Job  Does  Not  Require 

Group  Number  of  Respondents  That  Type  of  Data 

HAS  SEEN  BUT  NOT 

USED  WSSC  OUTPUT  41  22% 

HAS  NOT  SEEN 

WSSC  OUTPUT _ 47 _ 23% _ 

OVERALL  88  23% 


Of  the  88  respondents,  20  (23%)  said  that  their  job  does  not  re¬ 
quire  WSSC  type  data.  For  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  and 
the  HAS  NOT  SEEN  WSSC  OUTPUT  groups,  these  percentages  were  22%  and 
23%,  respectively. 

Not  all  the  respondents  who  circled  B  specified  their  reasons. 

Those  reasons  that  were  specified  are  given  in  the  following  unedited 
lists.  A  separate  list  is  provided  for  each  of  the  two  groups  considered. 


***********  *  * : 


******************************************************************* 


Question  13. 

the 

Office  of 

a . 

How  would 

Very 

b. 

How  would 

Very 

If  you  have  made  requests  for  WSSC  data  from 


5  Very  Good 


5  Very  Fast 


* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 


******************************************************************* 


WSSC  USERS  (N=55)  and  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  group 
(N=48)  were  instructed  to  answer  question  13  if  they  had  made  requests 
for  WSSC  data  from  the  Office  of  VAMOS C.  Of  the  103  possible  respondents 
65  (63%)  answered  part  (a)  and  62  (60%)  answered  part  (b) .  The  results 
are  summarized  below: 


Question 

Number  of 
Respondents 

Mean 

Rating 

Response 
2  3 

'  4 

5 

How  would  you  rate 
the  procedures  for 
obtaining  WSSC 
data? 

65 

3.57 

3% 

8% 

40% 

28% 

21% 

How  would  you  rate 
turnaround  time  for 
obtaining  reports? 

62 

3.58 

2% 

14% 

34% 

24% 

26% 

The  results  above  indicate  that  respondents  were  satisfied  with 


both  the  procedures  and  turnaround  time  for  obtaining  WSSC  reports. 


3.  There  are  so  many  "zero"  entries  in  the  WSSC  data 
I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded 

4.  Multi  sources  can  cause  variations  and  a/c 


G.  Depot  Installation  Support 

1.  The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

2.  During  evaluation  of  depot  costs  found  that  on  an 
average  these  value  combined  exceed  BACE  model 
results  by  about  50% 

3.  There  are  so  many  "zero"  entries  in  the  WSSC  data 
I  received  that  it  was  useless  since  1  know  there 
are  costs  which  should  have  been  recorded 

4.  Multi  sources  can  cause  variations  and  a/c 


H.  Medical  Care 


1.  The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

2.  This  value  seemed  to  vary  from  much  too  high  to 
much  too  low 

3.  Headcount 


I.  Permanent  Change  of  Station 

1.  The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

2.  This  value  seemed  too  high  I  would  like  to  know 
what  makes  it  up. 

3.  Headcount 


Rat ing 

Too  Low 

Too  High/ 
Too  Low 

Rating 

Too  High/ 
Too  Low 

Too  High 

Too  Low 

Too  High/ 
Too  Low 

Rating 

Too  High/ 
Too  Low 

Too  High/ 
Too  Low 

About  Right 

Rating 

Too  High/ 
Too  Low 

Too  High 
About  Right 
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E.  Depot  Maintenance  Rating 

1.  Compared  to  AFR  173-13  Too  High/ 

Too  Low 

2.  Incomplete  data  Too  Low 

3.  DMIF  oriented 

Do  not  capture  all  costs  Too  Low 

4.  The  systems  used  for  input  are  not  edited  for  accu-  Too  High/ 

racy.  The  quality  of  the  input  is  poor.  Too  Low 

5.  During  evaluation  of  depot  costs  found  that  on 

an  average  these  value  combined  exceed  BACE  model 

results  by  about  50%  Too  High 

6.  Actual  data  can  be  obtained  from  depot  managers. 

Your  costs  are  averages .  We  need  actual  cost  or  Too  High/ 

each  ACFT.  Too  Low 


Compared  favorably  with  G079  system  About  Right 

There  are  so  many  "zero"  entries  in  the  WSSC  data 
I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded  Too  Low 

Feeder  system  for  these  costs  is  H036C  (WSCRS), 

which  pro-rates  total  costs  in  these  categories  Too  High/ 

according  to  flying  hours  Too  Low 

Experience  comparing  with  other  sources  About  Right 

Average  over  8  years  of  data  About  Right 


12.  We  need  a  method  to  track  all  costs  by  weapon 

system  Too  Low 


F.  General  Depot  Support  Rat  in 

The  systems  used  for  input  are  not  edited  for  accu-  Too  High/ 
racy.  The  quality  of  the  input  is  poor.  Too  Low 

During  evaluation  of  depot  costs  found  that  on  an 
average  these  value  combined  exceed  BACE  model 
results  by  about  50%  Too  High 


c. 

Installation  Support 

Rating 

1. 

The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

Too  High/ 
Too  Low 

2. 

These  values  on  a  whole  compared  well  with  several 
BACE  model  results,  but  not  all. 

About  Right 

3. 

There  are  so  many  "zero"  entries  in  the  WSSC  data 

I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded. 

Too  Low 

4. 

Headcount 

About  Right 

D. 

Sustaining  Investment 

Rating 

1. 

Compared  to  AFR  173-13 

Too  Low 

2. 

Incomplete  data 

Too  Low 

3. 

Do  not  capture  all  parts  within  an  end  item.  Also 
reporting  by  master  stock  number.  Gives  misleading 
costs  by  NSN . 

Too  High/ 
Too  Low 

4. 

The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

Too  High/ 
Too  Low 

5. 

In  testing  against  BACE  model  results  this  value 
was  between  60%  to  90%  too  low. 

Too  Low 

6. 

There  are  so  many  "zero"  entries  in  the  WSSC  data 

I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded. 

Too  Low 

7. 

Feeder  system  for  these  costs  is  H036C  (WSCRS),  which 
pro-rates  total  costs  in  these  categories  according  Too  High/ 
to  flying  hours  Too  Low 

8. 

Experience 

About  Right 

9. 

Model  changes,  collection  sys. 

Too  High/ 
Too  Low 

10. 

We  need  to  track  these  costs  by  weapon  system 

Too  Low 

11. 

Contract  Logistics  Support  (CLS)  EEIC  385  for  C-12A 

in  FY82  was  $6.8M.  Of  this  amount  $4.4M  was  reimburs¬ 


able  and  $2.4M  chargeable  to  direct  Air  Force  D160, 
dated  3  May  83,  for  FY82  does  not  appear  to  reflect 
the  CLS  costs.  Too  Low 
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A.  Unit  Operations 

1.  The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

2.  These  values  on  a  whole  compared  well  with  several 
BACE  model  results,  but  not  all. 

3.  There  are  so  many  "zero"  entries  in  the  WSSC  data 
I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded. 

4.  Headcount 

5.  The  number  of  maintenance  manhours  per  flying  hour 
seems  to  reflect  "justification"  of  number  of  people 
on  board  rather  than  actual  MMH/FH. 


B.  Below  Depot  Maintenance 

1.  Compared  to  AFR  173-13 

2.  Does  not  utilize  existing  A/F  data  collection 
systems 

3.  The  systems  used  for  input  are  not  edited  for  accu¬ 
racy.  The  quality  of  the  input  is  poor. 

4.  These  values  on  a  whole  compared  well  with  several 
BACE  model  results,  but  not  all. 

5.  There  are  so  many  "zero"  entries  in  the  WSSC  data 
I  received  that  it  was  useless  since  I  know  there 
are  costs  which  should  have  been  recorded. 

6.  Systems  of  USAF  reporting  tend  to  under-report 
these  costs. VAMOSC  uses  those  systems 

7.  AFM  66-1  is  not  an  accurate  recording  system 


8.  Headcount 


Rating 

Too  High/ 
Too  Low 

About  Right 

Too  Low 
About  Right 

About  Right 

Rating 
Too  High 

Too  Low 

Too  High/ 
Too  Low 

About  Right 

Too  Low 

Too  Low 

Too  High/ 
Too  Low 

About  Right 
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Number  of  Don't  Too  About  Too  Sometimes  Too  High 


Q17 


i 


i 

Group 

Number  of 
Respondents 

Mean 

Response 

1 

Response 

2  3  4 

5 

• 

ALL  USERS 

33 

3.18 

0% 

24% 

36% 

36% 

3% 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

17 

3.47 

6% 

6% 

35% 

41% 

12% 

• 

HAS  NOT  SEEN  WSSC 
OUTPUT 

10 

3.10 

20% 

10% 

30% 

20% 

20% 

• 

OVERALL 

60 

3- 25 

5% 

17% 

35% 

35% 

8% 

• 

m 

Group 

Number  of 
Respondents 

Mean 

Response 

J_ 

Respons 
2  3 

e 

4 

5 

( 

ALL  USERS 

33 

2.67 

6% 

36% 

42% 

15% 

0% 

• 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

18 

2.83 

6% 

33% 

33% 

28% 

0% 

HAS  NOT  SEEN  WSSC 
OUTPUT 

10 

3.20 

o% 

Q% 

80% 

20% 

0% 

■ 

OVERALL 

61 

2.80 

5% 

29% 

46% 

20% 

0% 

• 

tf 

Q19 

Group 

Number  of 
Respondents 

Mean 

Response 

J. 

Respons 
2  3 

e 

4 

_5 

• 

ALL  USERS 

33 

3.06 

0% 

21% 

51% 

27% 

0% 

• 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

18 

3.22 

6% 

6% 

56% 

28% 

6% 

HAS  NOT  SEEN  WSSC 
OUTPUT 

9 

2.44 

22% 

11% 

66% 

0% 

0% 

. 

OVERALL 

60 

3.02 

5% 

15% 

55% 

23% 

2% 

« 

« 
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I 


The  responses  for  AF  WSSC  USERS  and  DOD  contractor  WSSC  USERS  are  sum- 


marized  below: 

Q16 

Number  of 

Mean 

Response 

WSSC  USERS 

Respondents 

Response 

2  3  4 

5 

Air  Force 

13 

2.85 

8% 

31%  31%  31% 

0% 

DOD  Contractor 

17 

3.24 

0% 

24%  29%  47% 

0% 

Number  of 

Mean 

Response 

WSSC  USERS 

Respondents 

Response 

l 

2  3  ± 

5 

Air  Force 

13 

3.00 

0% 

38%  23%  38% 

0% 

DOD  Contractor 

17 

3.47 

0% 

6%  47%  41% 

6% 

m 

WSSC  USERS 

Number  of 
Respondents 

Mean 

Response 

J. 

Respons 
2  3 

e 

4 

5 

Air  Force 

13 

3.00 

0% 

23% 

54% 

23% 

0% 

DOD  Contractor 

17 

2.47 

12% 

41% 

35% 

12% 

0% 

Q19 

WSSC  USERS 

Number  of 
Respondents 

Mean 

Response 

Response 

2  3  4 

5 

Air  Force 

13 

3.08 

0% 

15%  62% 

2  3% 

0% 

DOD  Contractor 

17 

3.06 

0% 

24%  47% 

29% 

0% 

It  should  be  noted  that  of  the  four  questions  pertaining  to  the 
users  manual,  question  19  incorporated  a  reverse  scale.  That  is,  the 


response  of  1,  rather  than  of  5,  indicated  the  best  rating. 


***************************************************************** 


Question  20.  How  would  you  improve  the  WSSC  users  manual? 


***************************************************************** 


Eighteen  of  the  61  respondents  who  indicated  they  had  read  or 
used  the  users  manual  had  suggestions  for  improving  the  manual.  Their 
unedited  suggestions  are  listed  below: 


1.  Make  it  easier  to  understand  and  use 


2.  Provide  more  up-to-date  examples  of  output  data  and  request 
forms  filled  out. 


3.  Add  ICBM  systems  -  as  soon  as  possible. 


4.  By  attempting  to  write  in  in  everyday  language-in  plain  English 


5.  I  would  leave  it  alone.  I  think  any  problems  arise  from  not  being 
able  to  directly  adapt  it  for  our  requirements  and  once  this  is 
done  it  will  be  easier  and  more  understandable. 


6.  Make  it  shorter! 


7.  Write  it  from  a  users  viewpoint  of  what  he  would  want  the  manual 
to  kick  out 


8.  The  manual  does  not  adequately  explain  what  a  particular  cost 

category  actually  means.  It  is  difficult  to  extract  the  precise 
limitations  of  a  particular  figure  from  all  the  abbreviations/ 
formulas.  A  clear,  but  complete,  explanation  of  each  field  is 
essential . 


-52- 


9.  If  the  current  manual  is  anything  like  the  draft  was  it  could 

stand  having  short  very  numerous  examples.  For  each  user  activity 
examples  should  begin  with  the  smallest  sensible  option  and  progress 
gradually  to  the  more  complex.  Index  tabs  for  the  various  chapters 
and  a  keyword  index  would  help.  Also,  there  should  be  no  broadside 
illust.  If  such  are  absolutely  needed  they  should  be  collected  in 
a  separate  Vol.  II  for  ease  of  reference. 

10.  Needs  more  explanation  of  what  the  data  (costs)  include. 


11.  Provide  a  good  summary  which  is  fairly  short 

12.  More  detail  description  of  the  cost  categories 


13.  Improve  readability 

14.  Provide  more  background  information  on  each  cost  category,  stating 
exactly  what  it  covers. 

Allow  independent  contractor  to  write  manual  who  was  actually  in¬ 
volved  in  the  WSSC  development.  This  provides  more  consistent 
information. 


15.  Show  more  examples  in  narrative 


16.  Provide  better  definitions 

17  Make  it  more  usable  for  someone  not  too  familiar  with  system 


18.  Give  examples 
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*************************************************************** 
*  * 

*  * 

*  * 

*  Question  21.  In  general,  are  you  satisfied  with  the  WSSC  * 

*  system?  * 

*  * 

*  Not  Satisfied  12345  Very  Satisfied  * 

*  * 

*  * 

*************************************************************** 


All  respondents  familiar  with  the  WSSC  system  (N=156)  were  in¬ 
structed  to  answer  question  21.  There  were  37  nonresponses  to  this 
question.  Virtually  all  of  these  nonresponses  (34  out  of  37)  were 
from  the  HAS  SEEN  BUT  NOT  USED  WSSC  OUTPUT  and  the  HAS  NOT  SEEN  WSSC 
OUTPUT  groups.  The  responses  are  summarized  below: 


Group 

Number  of 
Respondents 

Mean 

Response 

J. 

Response 
1  1 

4 

5 

WSSC  USERS 

52 

3.06 

6% 

13% 

54% 

23% 

4% 

HAS  SEEN  BUT  NOT  USED 
WSSC  OUTPUT 

35 

2.60 

9% 

23% 

68% 

0% 

0% 

HAS  NOT  SEEN  WSSC 
OUTPUT 

32 

2.53 

22% 

19% 

47% 

9% 

3% 

OVERALL 

119 

2.78 

11% 

18% 

55% 

13% 

3% 

The  responses  for  AF 

WSSC  USERS  and 

DOD  contractor  WSSC  USERS  are  sum- 

marized  below: 

WSSC  USERS: 

Number  of 
Respondent  s 

Mean 

Response 

Response 
2  3 

'  4 

5 

Air  Force 

30 

2.83 

10% 

23% 

44% 

20% 

3% 

DOD  Contractor 

18 

3.44 

0% 

0% 

61% 

33% 

6% 

As  can  be  seen  from  these  two  sets  of  data,  respondents  tended 
to  be,  on  the  average,  moderately  satisfied  with  the  WSSC  system.  With 
in  the  WSSC  USER  group,  the  DOD  contractor  users  gave  the  system  a  very 
positive  overall  evaluation.  (All  ratings  were  at  or  above  the  middle 
of  the  rating  scale.)  With  Air  Force  users,  on  the  other  hand,  there 
was  a  split  decision,  with  the  positive  evaluations  offset  by  negative 


ones . 


******************************************************************* 

* 

* 

$  Question  22.  Do  you  have  any  other  suggestions  for  improving 

* 

* 

* 

* 

******************************************************************* 


the  WSSC  system? 


All  respondents  familiar  with  WSSC  (N=156)  were  asked  to  answer 
question  22.  Forty-three  respondents  answered  this  question.  Their 
unedited  responses  are  listed  below: 

1.  Expand  data  base  to  include  tactical  weapons! 


2.  My  work  seems  to  require  CSCS  data  more  often  than  WSSC.  Weapon 
system  level  data  is  aggregated  at  a  high  level  and  it  is  not 
clear  what  causes  the  costs  to  be  high  or  low.  Cannot  judge 
sensitivity  of  costs  to  changes  in  reliability  or  changes  in 
maintenance  or  operating  concepts. 


3.  Need  software  support  cost  data! !!!!! (either  here  or  component 
cost  system)  Software  support  costs  are  an  ever-increasing  area 
of  concern,  and  VAMOSC  seems  to  be  slow  in  addressing  them. 


4.  Educating  potential  users  would  be  very  valuable. 

5.  Include  (2)  and  (3)  levels  as  noted  in  question  11,  i.e.,  WSSC 
data  on  missile  and  satellites/spare  system 

6.  Document  changes  to  program  over  time  on  how  you  are  improving 
it  so  you  can  better  understand  deviations. 

7.  Send  reports  when  they  are  ordered! 

8.  Establish  standard  cost  estimating  structure. 

9.  Include  aircraft  that  are  contractually  supported  even  though  all 
data  elements  are  not  present,  i.e.,  KC-10 

10.  Not  at  this  time.  May  have  some  once  we  start  using  WSSC  data  on 
a  regular  basis. 


9 
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********* 


11.  Let's  get  it  up  and  working  pronto 

12.  In  general,  we  are  more  interested  in  specific  failure  data,  usage 
rates,  etc.,  than  in  costs,  especially  bookkeeping  costs  like  man¬ 
power.  However,  acquisition  costs  of  specific  components  may  be 
of  interest  in  certain  applications. 

13.  As  you  know,  it  is  very  important  that  WSSC  provide  explanations 

of  what  the  costs  represent,  e.g.,  spares  currently  in  WSSC  are  not 
the  total  costs  the  Air  Force  must  budget  for.  This  must  be  known 
to  users  (especially  contractors). 

14.  Need  to  promote!!! 

15.  WSSC  has  a  great  need  to  be  made  more  accurate,  but  the  cost  may 
prohibit  such  action.  If  this  be  the  case  then  we  should  arrange 
for  review  each  time  existing  management  systems  are  changed  to 
see  that  accomodations  are  made  to  improve  WSSC  accuracy. 

16.  Data  should  be  in  constant  1967  dollars  (baseline  year  for  the  CPI) 

17.  Improve  input. 

18.  Provide  cost  categories  which  correspond  to  CORE  factors  and 
Tables  2-1,  2-2,  AFR  173-13. 

19.  No,  however  we  will  order  some  products  and  may  have  some  suggestions 
in  the  future.  We  had  ordered  some  products  in  the  past  but  the 
data  was  incomplete.  We  have  not  looked  at  any  products  recently. 

20.  A  training  course 

21.  Apply  to  C-E  systems 

22.  Might  try  starting  a  travelling  class  to  help  users  understand. 

A  two-week  course  in  what  WSSC  can  do  for  the  user  with  practical 
work  problems  would  be  very  valuable.  It  would  seem  a  shame  to 
waste  all  that  accumulated  data  because  its  too  complex  for  the 
average  user  or  not  in  format  he  wants. 


23.  As  with  any  system  extensive  work  on  accuracy,  editing  and  otherwise 
improving  inputs  would  be  dictated. 

24.  None  for  improvement,  but  would  like  to  learn  more  about  the  system. 
It  has  been  most  informative  with  respect  to  what  has  been  received 
to  date. 

25.  We  use  the  CSCS  system.  It  would  be  very  helpful,  in  fact  it  is 
essential;  except  it  is  obvious  that  the  data  is  very  inaccurate. 

Any  help  in  this  area  would  be  extremely  helpful. 

26.  I  plan  to  attend  the  Tri-Service  VAMOSC  Conference  in  May  to 
get  more  insight  and  understanding  of  the  system.  After  this 
I  may  have  suggestions. 


27.  Accuracy  of  the  WSSC  products  should  be  evaluated  by  a  group 
that  is  independent  yet  knowledgeable  about  the  WSSC  system. 

They  should  approach  the  task  as  if  they  were  going  to  use  the 
products  themselves  with  the  possibility  of  comparing  the  products 
with  other  sources. 

28.  The  biggest  problem  with  WSSC  is  not  the  system  but  the  data 
that  feeds  WSSC.  The  data  reporting  does  not  seem  to  be  com¬ 
plete  and  accurate.  Emphasis  should  be  put  on  data  collecting 
and  reporting. 


29.  Request  that  data  be  developed  and  users  manual  modified  to  provide 
the  attached  information 


30.  Needs  more  missionary  work  done  to  make  people  aware  of  capabilities 
Data  must  be  converted  into  information 


31.  Include  evaluation  factors  by  cost  element  by  year  for  5  yr  report 

32.  Include  tactical  and  strategic  missile; separate  parameters  by 
functional  categories 

33.  Make  costs  realistic.  Data  comes  from  unreliable  sources.  Expand 
to  include  data  from  CAMS  in  near  real-time  mode. 


34.  Currently  we  have  three  major  problems  in  the  Weapon  System 
Management  arena: 

(1)  Visibility  of  actual  requirements 

(2)  Funding  the  requirements  in  a  balanced  method  to  complement 
the  overall  system 

(3)  Making  the  procurement  cycle  respond  by  making  items  available 
when  needed. 

Any  analysis  tool  that  would  enhance  any  of  the  above  would  be 
of  great  value. 

35.  If  the  objective  is  to  show  upper  management  total  costs,  it 
appears  to  fulfill  this  requirement.  At  the  working  level, 

as  in  my  position,  detailed  rates  for  depot  costs  would  be  useful. 


36.  The  data  was  used  as  a  tool  to  cross-reference  work  unit  code  to 

NSN.  However,  it  is  far  from  complete  and  I  feel  that  once  complete 
the  VAMOSC  will  offer  a  source  for  WUC  to  NSN  cross  reference  that 
is  not  now  available.  I  wage  a  dedicated  effort  to  complete  this 
part  of  VAMOSC  because  it  is  a  valuable  tool  in  setting  up  inventure 
relationships  for  engine  analysis  with  regard  to  Dyna-Metric. 


37.  Definitely  need  to  include  Guard  and  Reserve  data 


38.  All  costs  per  aircraft  should  be  per  PAA  not  inventory  aircraft 
"Possessed  PAA" 

39.  The  system  appears  to  be  on  the  right  track,  it  is  maturing  and  has 
the  problems  associated  with  any  new  program-such  as  erroneous  WUC, 
missed  costs,  incorrect  reporting,  etc...  Once  the  system  gets  rid 
of  the  bugs,  it  should  be  useful  product. 


40.  GIGO-Try  putting  in  good  data 


41.  Add  automatic  test  equipment  and  common  electronic  support  equipment 
(FSC:6625)  in  the  model 


42.  Require  orientation  as  to  source  of  information  used  in  WSSC  and 
how  it  is  distributed 
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43.  Make  the  system  more  real  time.  Example:  We  need  to  know  O&S 
cost  on  the  KC-135E.  There  are  over  60  of  these  aircraft  flying 
but  WSSC  indicates  only  a  few.  The  system  must  be  at  least  6-12 
months  behind  actual  cost.  We  have  to  manually  track  O&S  cost 
because  your  system  is  not  timely. 


A************************************************* ******** *********** 


54%  Yes 


45%  No 


1  %  Don't  Know 


*  Question  23.  Would  a  system  which  reports  historical  operating  J 

* 
* 
* 
* 
* 
* 
* 
* 
* 
* 


$  and  support  costs  of  Air  Force  aircraft  be  useful  to  you  in 

$  your  work? 

* 

* 

* 

* 

* 

*  „ 
********************************************************************* 


All  respondents  who  answered  no  to  question  4,  i.e.,  said  they  were 
not  familiar  with  WSSC  system,  (N=137)  were  instructed  to  answer  question 
23.  There  were  no  nonresponses  to  this  question.  This  question  was 
used  to  classify  respondents  unfamiliar  with  WSSC  into  the  UNFAMILIAR 
WITH  WSSC:  POTENTIAL  USER  group  (if  they  answered  yes)  or  the  UNFAMILIAR 
WITH  WSSC:  NOT  POTENTIAL  USER  (if  they  answered  no  or  don't  know). 
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******************************************************* *************** 

* 

•k 

*  Question  24.  If  yes  [to  question  23],  how  valuable  would  it  be? 

* 

*  Slightly  Valuable  12345  Very  Valuable 

* 

********************************************************************* 


Only  those  respondents  who  answered  yes  to  question  23,  i.e., 
UNFAMILIAR  WITH  WSSC:  POTENTIAL  USER,  (N=74)  were  instructed  to  answer 
question  24.  There  were  three  nonresponses  to  this  question.  The 
responses  are  summarized  below: 


Number  of  Respondents 
71 


Mean  Response 
4.00 


Response 

1  2  3  4  5 

8%  3%  17%  24%  48% 


The  respondents  to  this  question  were,  on  the  whole, 
that  a  system  such  as  WSSC  would  be  of  much  value  in 


of  the  opinion 
their  work. 


I 


**■**■  ******** 


****************************************************************** 


Question  25.  Would  you  like  more  information  about  the  WSSC 
system? 


62%  Yes 


38%  No 


****************************************************************** 


Only  the  respondents  not  familiar  with  the  WSSC  system  ( N= 137) 
were  instructed  to  answer  question  25.  There  were  two  nonresponses 
to  this  question.  The  results  are  summarized  below: 


Group 

Number  of  Respondents 

Yes 

NOT  FAMILIAR  WITH  WSSC: 
POTENTIAL  USER 

72 

89% 

NOT  FAMILIAR  WITH  WSSC: 

NOT  POTENTIAL  USER 

63 

32% 

OVERALL 

135 

62% 

Because  respondent  name  and  address  was  optional  information,  not  all 
respondents  who  answered  "yes"  to  this  question  could  be  identified. 
However,  Desmatics  provided  the  Office  of  VAMOSC  with  a  list  of  those 
respondents  for  which  this  information  was  available. 
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While  the  present  study  represents  a  considerable  expenditure  of 
effort  in  an  attempt  to  identify  WSSC  users  and  potential  users,  and 
to  assess  the  level  of  satisfaction  with  WSSC,  it  should  not  be  con¬ 
sidered  the  final  word,  since  the  status  of  the  user  community  is  some¬ 
what  dynamic.  It  is  therefore  prudent  for  the  Office  of  VAMOSC  to 
make  periodic  assessments  of  user  satisfaction. 

Desmatics  recommends  that  the  Office  of  VAMOSC  consider  conducting 
a  follow-up  survey  in  approximately  three  years.  The  current  survey 
would  constitute  a  baseline  against  which  the  follow-up  could  be  com¬ 
pared  in  order  to  assess  changes  in  user  satisfaction. 

Office  of  uAMOSC  Comments:  "Concur.  The  Office  of  VAMOSC  anticipates 
that  the  WSSC  system  will  be  mature  as  of  the  FY88  timeframe.  At  that 
point  a  reappraisal  of  user  reactions/satisfaction  will  be  valuable." 


dents  expressed  a  desire  for  more  detail.  Desmatics  recommends  that 
the  Office  of  VAMOSC  consider  ways  to  meet  the  needs  for  more  detail 
that  some  readers  have  expressed. 


Office  of  VAMOSC  Comments;  "Concur.  A  requirement  has  been  specified 
to  redesign  all  customers'  manuals  (AFR  400-31,  Volumes  I-IV)  into  a 
more  "user  friendly"  format.  After  the  results  of  this  effort  are 
received,  all  manuals  will  be  rewritten.  Estimated  completion  date 
is  FY87." 

7.  Review  of  Survey  Respondents'  Comments 

Survey  respondents  provided  a  number  of  comments,  which  were  in¬ 
cluded  in  Section  IV.  Desmatics  recommends  that  the  Office  of  VAMOSC 
review  these  comments  to  assess  their  relevance  and  possible  impact. 

Office  of  VAMOSC  Comments:  "Concur.  These  comments  are  all  valuable 
and  will  be  used  as  support  for  several  needed  refinements." 


5.  Constant  Year  Dollars 


Currently  each  year's  WSSC  cost  data  is  reported  in  "then  year" 
dollars.  Several  survey  respondents  indicated  that  it  would  be  use¬ 
ful  or  essential  to  have  a  capability  to  get  data  for  two  or  more 
years  in  the  same  specified  constant  year  dollars.  This  could  be 
accomplished  for  interrogation  reports  by  suitable  modification  of  the 
report  generation  program,  which  would  not  require  changing  the  data 
base.  Desmatics  recommends  that  the  Office  of  VAMOSC  consider  making 
such  an  enhancement . 


Office  of  VAMOSC  Comments:  "Do  not  concur.  This  enhancement  could 
indeed  be  useful  to  some  analysts.  However,  it  is  not  essential  to  the 
utility  of  VAMOSC  data  and  it  would  require  a  separate  generation  for 
each  user  due  to  varying  base  years  and  inflation  rates.  There  are  in¬ 
sufficient  resources  available  to  include  this  feature." 


6.  Users  Manual  Revision 


While  survey  respondents  generally  found  the  WSSC  users  manual 
(AFR  400-31,  Vol.  II)  understandable,  useful  and  fairly  easy  to  read, 
there  were  indications  that  for  some  segments  of  the  user  manual  audience, 
it  could  be  improved.  Specifically,  a  relatively  large  number  of  respon- 


Ideally  a  WSSC  user  should  read  the  manual  to  find  out  what  costs 
are  included  in  each  category,  what  data  sources  are  used,  how  costs 
are  allocated,  and  what  the  capabilities  and  limitations  of  the  system 
are.  However,  there  are  two  problems:  (1)  many  users  do  not  read 
the  manual  and  thus  may  not  be  fully  aware  of  these  aspects,  and  (2) 
there  is  usually  a  time  lag  in  updating  manuals  which  would  result  in 
the  user  not  having  timely  notice  of  changes  to  the  system. 

Desmatics  recommends  that  the  Office  of  VAMOSC  make  essential  user 
information  a  part  of  every  report  product  disseminated  by  the  WSSC 
system.  This  could  be  accomplished  by  having  the  report  generator 
program(s)  produce  a  section  of  text  which  precedes  the  actual  data. 
This  text  should  summarize  the  essentials  with  respect  to  costs  in¬ 
cluded,  data  sources,  allocation  methods  and  other  features.  Signifi¬ 
cant  changes  made  in  any  of  these  areas  should  be  highlighted  for 
emphasis.  Such  material  would  be  comparatively  easy  to  update  as 
required.  This  type  of  information  should  also  be  disseminated  as  a 
separate  entitity  to  current  and  potential  WSSC  users,  perhaps  through 
the  newsletter  mentioned  in  a  previous  section. 

Office  of  VAMOSC  Comments:  "Concur  in  part.  After  development  of  the 
System  Logic/Event  History  portrayal  capability  and  the  End  Item  Event 
History  portrayal  capability,  this  recommendation  will  be  reevaluated," 


-74- 


3.  Software  Maintenance  Cost  Visibility 


When  asked  to  assess  the  need  for  WSSC  to  provide  cost  visibility 
for  software  maintenance,  a  large  number  of  respondents  indicated  this 
would  be  very  useful  or  essential.  Desmatics  recommends  that  the  Office 
of  VAMOSC  examine  the  possibility  of  providing  separate  visibility  for 
software  maintenance  costs.  This  would  enhance  WSSC  cost  category  com¬ 
parability  with  CORE,  a  topic  which  was  discussed  in  the  previous 
section. 

Office  of  VAMOSC  Comments:  "Concur.  This  is  mandated  by  the  CAIG  guide 
lines  currently  in  effect.  The  inclusion  of  these  costs  is  contingent 
upon  the  development  of  a  data  system  to  portray  them.  The  requirement 
has  been  specified  and  is  awaiting  funding." 


4.  Cost  Category  Identification 

Many  survey  respondents  who  had  used  WSSC  had  not  read  the  Users 
Manual  (AFR  400-31,  Volume  II).  This  may  partially  explain  the  need 
expressed  by  respondents  to  have  WSSC  reports  provide  optional  brief 
descriptions  of  the  costs  which  are  included  in  each  cost  category. 


2.  Cost  Category  CORE  Model  Compatabilitj 


User  survey  respondents  indicated  a  desire  for  more  comparability 
of  WSSC  cost  categories  with  those  of  the  CORE  model,  as  set  forth 
in  AFR  173-13.  While  many  of  the  categories  used  by  WSSC  have  direct 
correspondence  with  CORE,  respondents  indicated  that  more  comparability 

is  desired.  The  major  lack  of  compatability  between  WSSC  and  CORE 
occurs  in  two  areas,  software  maintenance  costs  and  personnel  acqui¬ 
sition  and  training  costs.  WSSC  was  designed  to  provide  visibility  for 
acquisition  and  training  costs,  but  the  collection  and  display  of  these 
costs  has  not  yet  been  implemented.  Software  maintenance  costs  are 
addressed  in  the  following  section. 

Desmatics  recommends  that  the  Office  of  VAMOSC,  as  part  of  an 
effort  to  increase  awareness  with  WSSC  (see  recommendation  No.  1), 
should  point  out  the  differences  between  the  historical  actual  O&S 
costs  of  WSSC  and  the  CORE  estimated  cost  factors.  It  is  conceivable 
that  as  WSSC  becomes  more  well  known,  many  users  of  the  CORE  model 
will  opt  instead  for  WSSC. 

Office  of  VAMOSC  Comments:  "Concur  in  part.  The  first  priority  of  the 
Office  of  VAMOSC  with  regard  to  AFR  173-13  and  CORE  is  to  align  VAMOSC 
data  elements  with  AFR  173-13  requirements.  From  that  point  VAMOSC 
can  feed  the  cost  factor  building  process  and  become  a  part  of  AFR  173-13 
rather  than  something  to  compare  it  with." 


this  is  a  "public  relations"  problem.  Support  for  the  WSSC  system  can 
best  be  nurtured  by  publicizing  its  availability  and  demonstrating  that 
it  provides  information  useful  to  a  large  segment  of  the  aircraft  O&S 
cost  community. 

Desmatics  recommends  that  the  Office  of  VAMOSC  implement  a  plan 
to  increase  familiarity  with  and  understanding  of  the  WSSC  system.  Such 
a  plan  might  include  the  following: 

(1)  A  promotional  package  for  potential  users  which  explains 
what  WSSC  is,  the  uses  for  WSSC  data,  how  WSSC  compares 
with  O&S  cost  models  (such  as  CORE),  examples  of  output 
and  usage,  and  how  to  get  data  and  additional  information 
on  the  system. 

(2)  A  newsletter  sent  to  those  who  have  requested  WSSC  data 
to  keep  them  informed  of  changes  and  enhancements  to 
the  system. 

(3)  Briefings  for  users  and  potential  users  which  explain 
the  WSSC  system  and  demonstrate  potential  uses  of  the 
data. 

Desmatics  also  recommends  that  the  Office  of  VAMOSC  contact  those  respon¬ 
dents  to  the  survey  who  gave  their  names  and  indicated  they  would  like 
more  information  on  WSSC. 

Office  of  VAMOSC  Comments:  "Concur.  A  promotional  package  is  being 
developed  which  includes  a  video  presentation  and  an  executive  overview. 
The  newsletter  will  be  nonrecurring  as  a  ban  on  new  recurring  publications 
is  in  effect.  VAMOSC  will  continue  to  solicit  opportunities  for  itself 
and  relevant  contractors  to  make  presentations  and  briefings  at  every 
possible  opportunity." 


that  many  of  them  intended  to  use  WSSC  in  the  future  or  are  currently 
indirect  users  (i.e.,  supervisors)  supported  by  others  who  do  use  WSSC. 

B .  RECOMMENDATIONS 


The  WSSC  system  will  continue  to  grow  in  acceptance  and  support  only 
to  the  extent  that  it  has  a  reputation  for  providing  accurate,  complete, 
timely,  pertinent  and  unbiased  information.  Not  only  must  WSSC  possess 
these  attributes  in  good  measure,  but  also  it  must  be  so  perceived  by 
its  users  and  by  the  aircraft  cost  community  in  general.  The  results 

of  the  present  study  suggest  that  while  user  acceptance  of  WSSC  can 
on  balance  be  considered  good,  there  are  a  few  areas  in  which  improve¬ 
ments  could  be  made. 

Based  on  the  results  of  the  user  survey,  Desmatics  has  formulated 
eight  recommendations  for  consideration  by  the  Office  of  VAMOSC.  These 
are  briefly  discussed  in  the  following  paragraphs.  Also  included  are 
the  comments  of  the  Office  of  VAMOSC  with  respect  to  Desmatics'  recom¬ 
mendations  . 

1.  Increasing  User/Potential  User  Awareness 

Responses  to  the  survey  indicate  the  existence  of  a  considerable 
number  of  potential  users.  Desmatics  believes  many  of  them  would,  in 
fact,  use  the  WSSC  system  if  they  were  made  aware  of  its  existence  and 
capabilities.  In  addition,  some  current  users  (particularly  within 
the  Air  Force)  are  relatively  unfamiliar  with  the  system's  capabilities 
and  appear  to  underutilize  WSSC  as  a  source  of  data.  To  a  large  extent 
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WSSC  system  without  at  least  some  reference  to  the  users  manual  may 
result  in  misinterpretations  and/or  unintended  uses  of  the  data.  In 
fact  it  might  be  hypothesized  that  this  lack  of  reference  to  the  users 
manual  would  explain,  to  some  degree,  the  low  level  of  familiarity  with 
the  WSSC  system  within  the  AF  user  group. 

Even  among  those  users  who  have  read  the  users  manual  the  dichotomy 
exists,  with  the  AF  user  group  regarding  it  as  less  understandable  (Q16) 
and  less  useful  (Q 1 7 )  than  the  contractor  group.  Furthermore,  the 
majority  of  DOD  contractor  users  feel  that  there  should  be  more  detail 
in  the  manual,  whereas  the  majority  of  AF  users  tend  to  be  relatively 
satisfied  with  the  level  of  detail  (Q18) . 

2.  Nonusers 

Of  the  293  survey  respondents,  238  do  not  use  the  WSSC  system. 

Of  these,  137  are  not  familiar  with  the  system.  Of  these  latter 
respondents,  54%  indicated  that  a  system  which  reports  historical 
O&S  costs  of  Air  Force  aircraft  (i.e.,  a  WSSC-type  system)  would  be 
useful  in  their  work  (Q23).  Thus,  there  appears  to  be  a  reasonably 
sized  group  of  potential  users  who  are  not  familiar  with  the  WSSC 
system. 

Of  the  101  nonusers  who  were  familiar  with  WSSC,  88  responded 
to  a  question  asking  -hy  they  had  not  used  WSSC  data  (Q14).  From 
this  group,  20  indicated  that  their  job  did  not  require  that  type  of 
data.  Of  the  64  respondents  who  specified  other  reasons,  it  is  evident 


The  most  important  question  as  to  how  the  WSSC  system  is  viewed 
was  Q21,  which  asked  respondents  to  rate  their  overall  level  of  satis¬ 
faction  with  WSSC.  Those  WSSC  users  who  were  DOD  contractors  gave 
the  system  a  very  positive  overall  evaluation.  (All  ratings  were  at 
cr  above  the  middle  of  the  rating  scale.)  For  AF  users,  on  the  other 
hand,  there  was  a  split  decision,  with  the  positive  evaluations  from 
some  respondents  offset  by  negative  ones  from  others.  Based  on 
this  question,  WSSC  could  be  judged  to  be  a  success  for  DOD  contrac- 
toi  users,  but  only  a  limited  success  for  AF  users. 

Thus,  with  respect  to  overall  satisfaction  with  WSSC,  there  appears 
to  be  a  somewhat  surprising  dichotomy  between  AF  users  and  DOD  con¬ 
tractor  users.  This  dichotomy  becomes  more  evident  as  responses  to 
other  questions  are  examined.  It  is  extremely  interesting  (and  some¬ 
what  disconcerting)  to  find  that,  as  a  group,  AF  WSSC  users  when  com¬ 
pared  to  DOD  contractor  users: 

a.  do  a  smaller  number  of  tasks  for  which  WSSC  could  be 
helpful  (Q3) , 

b.  regard  themselves  as  less  familiar  with  WSSC  (Q5), 

c.  find  the  WSSC  output  less  understandable  (Q7), 

d.  use  WSSC  for  a  smaller  number  of  tasks  (Q8) , 

e.  find  WSSC  less  useful  in  their  work  (Q9) , 

and  f.  have  a  lower  percentage  who  have  read  the  User's 
Manual  (Q15), 

Of  the  above  findings  perhaps  the  most  striking  is  the  last  one 
(f),  because  61%  of  the  AF  users  of  the  WSSC  system  have  never  read  or 
used  the  users  manual,  compared  to  6%  of  the  DOD  contractor  users. 

This  is  a  potentially  serious  problem,  since  any  attempts  to  use  the 


Eleven  WSSC  system  additions  or  changes  suggested  during  survey 
development  were  incorporated  into  the  survey  (Q 1 1 )  for  evaluation 
by  respondents.  Four  of  these  suggestions  stand  out  in  terms  of  the 
amount  of  support  that  was  indicated.  These  are: 

(1)  Provide  cost  categories  which  correspond  to  the  CORE 
factors  (AFR  173-13). 

(2)  Include  an  optional  brief  description  of  what  costs 
are  included  in  each  category  with  cost  reports. 

(3)  Provide  separate  visibility  of  software  maintenance 
costs . 

(4)  Add  capability  to  get  data  for  two  or  more  years  in 
the  same  specified  constant  year  dollars. 

These  suggestions  form  the  basis  for  some  of  the  recommendations  dis¬ 
cussed  in  Section  B. 

For  the  remainder  of  this  discussion,  it  is  useful  to  examine 
current  WSSC  users  and  nonusers  as  two  separate  groups.  Users  repre¬ 
sent  "customers"  whose  satisfaction  is  of  interest  to  the  Office  of 
VAMOSC.  The  nonuser  group  may  be  expected  to  contain  a  number  of 
potential  users,  whose  characteristics  are  also  of  interest. 

1.  WSSC  Users 

Survey  responses  were  obtained  from  55  users  of  the  WSSC  system. 
The  overwhelming  majority  were  Air  Force  employees  (33)  or  DOD  contrac 
tors  (18).  Their  responses  provided  information  on  a  number  of  topics 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  information  obtained  through  this  user  survey  should  prove  of 
value  to  the  Office  of  VAMOSC  in  assessing  the  comparative  strengths 
and  weaknesses  of  the  WSSC  system,  as  well  as  in  deciding  which  pos¬ 
sible  system  enhancements  should  be  planned.  It  must  be  realized,  how¬ 
ever,  that  because  the  WSSC  system  has  been  implemented  only  recently, 
user  evaluations  may  change  as  the  system  matures.  Thus,  the  present 
survey  may  also  serve  as  a  baseline  for  comparison  of  user  acceptance 
and  satisfaction  as  WSSC  evolves. 

Based  on  the  results  of  the  survey,  Desmatics  has  reached  a  number 
of  conclusions,  which  are  given  in  subsection  A.  Recommendations  based 
on  these  conclusions  are  discussed  in  subsection  B. 

A.  CONCLUSIONS 

In  general,  the  assessments  of  the  WSSC  system  reflected  in  the 
survey  responses  were  positive,  but  not  overwhelmingly  so.  This  is 
not  surprising  in  view  of  the  fact  that  the  survey  respondents  indi¬ 
cated  a  low  level  of  familiarity  with  WSSC.  For  example,  40%  of  those 
respondents  who  have  used  the  WSSC  system  in  their  work  rated  them¬ 
selves  in  the  lowest  two  of  the  five  levels  of  WSSC  familiarity  (Q5) . 
Further  evidence  of  low  familiarity  is  given  by  the  fact  that  only  a 
few  respondents  expressed  any  opinion  (positive  or  negative)  about  the 
accuracy  of  the  costs  produced  by  WSSC  (Q12). 


7.  What  I  think  would  help  exploit  WSSC  would  be  roving  teams  of  WSSC 
experts  who  would  go  to  an  installation  and  show  potential  users 
on  a  day-to-day  basis  (say  for  2  months)  how  use  of  WSSC  can  help 
them  do  their  work  smarter.  Generally  people  who  are  pressed  don't 
have  time  to  learn  anything  new.  A  visiting  team  would  help  provide 
that  time  by  making  parallel  studies  based  on  WSSC.  Results  would 
then  be  compared. 

8.  I  do  not  use  WSSC,  however  I  do  use  CSCS.  This  data  would  be  more 
valuable  if  it  were  more  accurate.  (Work  unit  codes  reported  on 
which  are  not  applicable  to  the  weapon  system;  work  unit  codes  not 
reported  on;  FSNs  not  matching  components,  etc. 


9.  Life  cycle  (constant  dollar)  costs  per  deployed  aircraft  will 
often  approximate  a  saw  tooth  wave  form  (sharp  rise  to  gradual 
decline  to  reasonable  stability  to  phase  out  rise) .  Distortion 
occurs  when  major  modifications  occur,  when  missions  are  changed, 
and  when  fleet  life  extension  decisions  are  made.  History  costs 
are  not  reliable  in  a  predictive  mode  unless  we  consider  where  we 
are  on  this  "wave  form". 


10.  Need  some  classes  for  the  users-possibly  on  AFIT  course  or  some 
AF  Technical  School  course.  Publish  newsletter  about  problems 
or  achievements  in  data  reporting  -  goals  with  ECD  for  achieving 
the  goals  -  give  examples  of  how  reports  are  being  used. 

11.  In  general  it  covered  most  areas  in  which  VAMOSC  needs  reworking 
or  change.  There  should  be  more  questions  on  getting  VAMOSC 
products  to  users  other  than  just  question  13  -  should  be  questions 
on  what  users  would  like  to  see  in  way  of  getting  products  and  their 
form  (fiche,  print-out,  bound  xerox  print-out).  I  would  like  to 
see  a  direct  access,  on-line  data  base  accessible  to  USAF  users 

by  computer.  There  should  alos  be  a  general  question  at  end  of 
survey:  would  any  answers  given  above  change  if  changes  are  made 
in  VAMOSC?  Why? 

12.  From  histori:al  basis  it  would  be  beneficial  to  have  info  to  defend 
POM  submissions  regarding  benefit  of  spares  dollars  to  enhance 
sustainability. 
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******************************************************************* 


J  Question  26.  Please  write  below  your  reactions  to  this  ques-  J 
J  tionnaire  (e.g..  Were  the  questions  ambiguous?,  Did  it  ask  J 
J  questions  which  you  feel  are  important?,  etc.)  J 

*  * 

*  * 

******************************************************************* 


All  respondents  were  instructed  to  answer  question  26.  Forty-four 
percent  of  the  293  respondents  answered  this  question.  Listed  below 
are  only  those  comments  that  pertained  to  the  WSSC  system. 


1.  Tracking  the  real  cost  of  an  airborne  mission  would  be  extremely 
valuable  in  justifying  many  support  equipment  and  acft.  subsystem 
R&D  programs.  The  military  can  save  millions  by  timely  modifications 
and  updates  of  their  equipment.  You  asked  the  questions  1  need 
answers  to. 


2.  You  could  have  done  more  to  find  out  what  our  data  requirements 
really  are.  Getting  reliable,  historical  data  on  parts  failures, 
usage  and  costs  is  an  overwhelming  problem  for  the  analysis 
community  and  there  are  always  innumerable  reasons  why  you  can't 
use  the  data-mostly  because  of  inability  to  separate  the  impact 
of  weapon  system  changes,  modifications  and  variations  in  role, 
mission,  and  utilization  rates. 


3.  What  was  meant  by  "aircraft".  I  work  with  avionics  systems  and 
information  of  the  type  indicated  in  Question  23  would  valuable 
if  it  were  for  avionics  systems. . .entire,  undifferentiate-  aircraft 
costs  would  not. 


4.  With  little  knowledge  of  WSSC  system  it  is  difficult  to  evaluate 
usefulness.  The  DCS/DS  function  is  to  provide  supply  support 
to  all  weapon  systems.  Often  data  is  needed  to  evaluate  the 
impacts  on  DS  of  weapon  system  changes  (chief  of  which  is  manpower 
impacts;  workload  impacts) . 


5.  O&S  costs  would  most  likely  be  useful  if  they  were  expressed  in 
terms  of  "cost  per  flying  hour." 


6.  Questions  were  clear  and  understandable  -  if  WSSC  can  provide 

a  cross  check  for  our  O&S  estimates  reasonability,  all  the  better. 


VI.  P^ZivJiNCES 


1.  Office  of  VAMOSC,  "Subsystem  Specification  of  the  Preprocessor 
(VAMOH),"  Department  of  the  Air  Force,  1981. 

2.  U.S.  Air  Force,  Air  Force  Regulation  400-31,  Volume  II,  "Visibility 
and  Management  of  Operating  and  Support  Cost  Program,  Weapon  System 
Support  Cost  (WSSC),"  1982. 

3.  U.S.  Air  Force,  Air  Force  Regulation  400-31,  Volume  III,  "Visibility 
and  Management  of  Operating  and  Support  Cost  Program,  Ground  Com- 
munications-Electronics  (C-E),"  1982. 

4.  U.S.  Air  Force,  Air  Force  Regulation  400-31,  Volume  IV,  "Visibility 
and  Management  of  Operating  and  Support  Cost  Program  (VAMOSC), 
Component  Support  Cost  System  (CSCS),"  1982. 
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APPENDIX  A 

i 


The  following  page  is  a  copy  of  the  letter  sent  by  HQ  USAF/LEYM 
to  locate  additional  users/potential  users  within  the  AF  community. 


i 
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» 


» 


REPLY  TO 
ATTN  OF: 


LEYM 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  UNITED  STATES  AIR  FORCE 
WASHINGTON,  D.C. 


subject  Weapon  System  Support  Cost  System  (WSSC) 


1 .  I  need  your  help  collecting  names  of  those  people  who  are 
current  or  potential  users  of  Weapon  System  Support  Cost 
(WSSC, DSD  D160.)  system  data. 

2.  To  understand  who  is  using  WSSC  data,  the  Chief  of  the  WSSC 
program  office  tasked  Desmatics,  Inc.,  to  identify  current  and 
potential  WSSC  data  users.  To  help  Desmatics,  Inc.  do  this,  I 
need  you  to  write  below  the  names  of  the  people  you  work  with 
(e.g.,  in  your  office,  other  offices,  contractors,  etc.)  who 
use,  or  could  possibly  use,  WSSC  data.  Please  return  this  sheet 
in  the  enclosed  envelope  as  soon  as  possible,  but  not  later  than 
12  March  84.  In  case  you're  not  familiar  with  the  WSSC  system, 
I've  enclosed  a  brief  summary  of  WSSC  and  a  form  for  requesting 
a  WSSC  data  sample. 

3.  I  appreciate  any  help  you  may  be  able  to  give  Desmatics, 

Inc.  If  you  need  more  information,  call  Arlene  Munson  or 
Jonathan  Levine,  Desmatics,  Inc.,  814-238-9621. 


3  Atch 

1 .  WSSC  Overview 

2.  Product  Request  Form 

3.  Return  Envelope 

Name  Mailing;  Address  Phone  Number 


9^ 
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JOHN  A.  JANISIESKI 

DEP  CHIEF.  MAINTENANCE  POLICY  DU 
D.  MAINTENANCE  &  Sl'PPLY.  DCSvL&E 


□  This  office  is  not  aware  of  any  current  or  potential  WSSC  users 


APPENDIX  B 


A  copy  of  the  survey  used  in  this  study  is  given  in  the  following 


pages . 


I 


The  HQ  AFLC/MML(VAMOSC) 

Weapon  System  Support  Cost  (WSSC)  System 
User  Survey 


The  purposes  of  this  survey  are  to  determine  WSSC  user  satisfaction, 
to  get  suggestions  for  improvements  or  modifications  to  the  WSSC  system, 
and  to  identify  potential  WSSC  users.  The  VAMOSC  (Visibility  and 
Management  of  Operating  and  Support  Cost)  System  consists  of  three 
separate  subsystems: 


(1)  WSSC,  the  Weapon  System  Support  Cost  System, 

(2)  C-E,  the  Ground  Communications-Electronics  System, 

(3)  CSCS,  the  Component  Support  Cost  System. 


This  survey  is  intended  to  collect  opinions  only  about  the  WSSC 
component  of  VAMOSC.  If  you  have  any  questions  about  this  survey, 
please  contact: 


Patricia  Weber  or  Jonathan  Levine 
Desmatics,  Inc. 

P.0.  Box  618 

State  College,  PA  16804 
(814)238-9621 


Address 


Optional 


Who  is  your  current  employer? 


A.  Air  Force 

B.  Army 

C .  Navy 


D.  Dept,  of  Defense 

E.  Dept,  of  Defense  contractor 

F.  Other,  please  specify: 


Which  of  the  following  tasks  involving  aircraft  do  you  perform  in 
your  work?  Please  circle  all  that  apply. 

A.  My  work  does  not  involve  aircraft 

B.  Life  cycle  cost  management 

C.  Trade-off  analysis 

D.  Budget  preparation 

E.  Life  cycle  cost  modeling/forecasting 

F.  Reliability/maintainability  studies 

G.  Logistics  forecasting/management 

H.  Manpower  forecasting/management 

I.  Systems  comparison  (existing  or  conceptual) 

J.  DSARC  submissions 

K.  Evaluate  product  performance  agreements  (warranties, 
maintenance  agreements,  etc.) 

L.  War  readiness  assessments 

M.  POM  submissions 

N.  Supportability  analysis 

O.  Readiness/sustainability  analysis 

P.  Other(s),  please  specify: 


Are  you  familiar  with  the  Air  Force  WSSC  system? 

□  Yes  I  1  No  (If  no,  please  go  to  question  23) 

How  familiar  are  you  with  the  WSSC  system?  Circle  the  number  on 
the  scale  which  best  represents  your  opinion. 


Slightly  Familiar  1  2  3  A  5  Very  Familiar 


6.  Have  you  ever  seen  WSSC  output  or  used  WSSC  data  in  your  work? 


□  Yes  No  (If  no,  go  to  question  14)  > 

7.  How  understandable  is  the  WSSC  output? 

Very  Difficult  12345  Very  EasY 

to  Understand  to  Understand  > 


8.  Please  circle  any  of  the  following  tasks  for  which  you  have  used 
WSSC  data. 

A.  I  have  not  used  WSSC  data  (go  to  question  10) 

B.  Life  cycle  cost  management 

C.  Trade-off  analysis 

D.  Budget  preparation 

E.  Life  cycle  cost  modeling/forecasting 

F.  Reliability /maintainability  studies 

G.  Logistics  forecasting/management 

H.  Manpower  forecasting/management 

I.  Systems  comparison  (existing  or  conceptual) 

J.  DSARC  submissions 

K.  Evaluate  product  performance  agreements  (warranties, 
maintenance  agreements,  etc.) 

L.  War  readiness  assessments 

M.  POM  submissions 

N.  Supportability  analysis 

O.  Readiness/sustainability  analysis 

P.  Other(s),  please  specify: 


9.  How  useful  is  WSSC  data  in  your  work? 

Not  Useful  12345  Very  Useful 

10.  Do  you  have  a  need  for  aircraft  operating  and  support  cost  data 
reported  at  any  of  the  following  levels?  Please  circle  all  that 
apply. 


A. 

Worldwide 

F. 

Mission  Design 

B. 

Command 

G. 

Mission 

C. 

Base 

H. 

Don't  know 

D. 

Squadron 

I. 

Other,  please  specify: 

E. 

Mission  Design  Series 
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The  following  suggestions  have  been  made  for  additions  or  changes 
to  the  WSSC  system.  Please  rate  the  usefulness  of  these  suggestions 
as  follows: 

O-Not  Useful  1-Somewhat  Useful  2-Very  Useful 
3- Essential  4-Don' t  Know 

A.  □  Change  base  level  maintenance  cost  categories  from 

organizational  categories  (i.e.,  squadrons)  to 
functional  categories  (e.g.,  airframe,  engine,  etc.) 

B.  □  Add  an  optional  narrative  description  of  aircraft 

history  and  major  changes  (new  acquisitions,  major 
modification,  etc.) 

c.  □  Add  capability  to  get  data  for  two  or  more  years 
in  the  same  specified  constant  year  dollars 

D.  □  Include  reports  on  strategic  missiles 

E.  □  Include  reports  on  spacecraft/space  systems 

F.  □  Include  an  optional  cumulative  description  of  all 

changes  to  WSSC  system 

G.  □  Expand  system  to  include  acquisition  costs 

H.  □  Provide  separate  visibility  of  software  maintenance 

costs 

I-  □  Add  costs  for  Air  Force  Reserve  and  Air  National 
Guard  aircraft 

J.  □  Include  an  optional  brief  description  of  what  costs 

are  included  in  each  category  with  cost  reports 

K.  □  Provide  cost  categories  which  correspond  to  the 

CORE  factors  (AFR  173-13) 

L.  Other  suggestion,  please  specify: 


M.  □  Other  suggestion  ,  please  specify: 


N.  Q  Other  suggestion,  please  specify: 


12.  Please  rate  the  accuracy  of  the  costs  produced  by  the  WSSC  system 
for  each  of  the  following  cost  categories. 


0-Don' t  know  1-Too  Low  2-About  Right 
3-Too  High  4-Sometimes  Too  High,  Sometimes  Too  Low 

Cost  Category  Reason  for  Accuracy  Rating 

|  |  Unit  Operations 

□  Below  Depot  Maintenance 

f  |  Installation  Support 

1  I  Sustaining  Investment 

|  |  Depot  Maintenance 

f  |  General  Depot  Support 


[  1  Depot  Installation  Support 


□  Med  ical  Care 


|  |  Permanent  Change  of  Station 
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13.  If  you  have  made  requests  for  WSSC  data  from  the  Office  of  VAMOSC: 
a.  How  would  you  rate  the  procedures  for  obtaining  WSSC  data? 


Very  Bad  12345  Very  Good 
b.  How  would  you  rate  turnaround  time  for  obtaining  reports? 
Very  Slow  12345  Very  Fast 


14.  If  you  have  not  used  WSSC  data,  please  specify  why. 

A.  Job  does  not  require  that  type  of  data.  § 

B.  Other,  please  specify: 


15.  Have  you  ever  read  or  used  the  WSSC  users  manual  (AFR  400-31  Vol.  II)? 

□  Yes  □  No  (If  no,  please  go  to  question  21) 

16.  How  understandable  is  the  WSSC  users  manual? 

Very  Hard  1  2  3  4  5  ^ery  Easy 

to  Understand  to  Understand 

17.  How  useful  is  the  WSSC  manual? 

Not  Useful  12345  Very  Useful 

18.  How  would  you  rate  the  level  of  detail  in  the  WSSC  users  manual? 

Not  Enough  .  „  -  ,  Too  Much 

Detail  Detail 

19.  How  easy  is  the  WSSC  users  manual  to  use? 

Very  Easy  12345  Very  Difficult 


» 


§ 
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20.  How  would  you  Improve  the  WSSC  users  manual? 


21.  In  general,  are  you  satisfied  with  the  WSSC  system? 

Not  Satisfied  12345  Very  Satisfied 

22.  Do  you  have  any  other  suggestions  for  improving  the  WSSC  system? 
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23.  Would  a  system  which  reports  historical  operating  and  support 
costs  of  Air  Force  aircraft  be  useful  to  you  in  your  work? 

□  Yes  □  No  □  Don't  know 

24.  If  yes,  how  valuable  would  it  be? 

Slightly  Valuable  12345  Very  Valuable 

25.  Would  you  like  more  information  about  the  WSSC  system? 

□  Yes  Q  No 

26.  Please  write  below  your  reactions  to  this  questionnaire  (e.g.. 
Were  the  questions  ambiguous?,  Did  it  ask  questions  which  you 
feel  are  important?,  etc.) 
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